HPC Systems Administrator, Slurm

You administer the batch scheduling and software layer of GPU clusters for training and simulation work. You configure Slurm partitions, accounting and fair-share, run node health checks and keep the MPI and container runtimes working for the people who submit jobs.

Team: GPU cluster deployment, Kind: Full-time, Location: Remote, United States, Pay: $99,000 to $159,000 a year

Duties

  • Configure Slurm partitions, quality of service levels and limits to match how each team runs its jobs.
  • Set up job accounting and fair-share, so usage is recorded and shared as agreed.
  • Write and run node health checks that drain a node before a faulty GPU, link or disk fails a job.
  • Build and maintain the MPI libraries, GPU libraries and container runtimes that jobs depend on.
  • Help users debug failed or slow jobs, and turn common fixes into documentation.
  • Plan scheduler and software upgrades, and test them before they reach users.
  • Report cluster use and queue wait times to the teams that share each cluster.
  • Automate node setup and configuration with Ansible or similar tools.

What you bring

  • 3+ years administering Linux HPC clusters.
  • Hands-on Slurm administration, including partitions, accounting and fair-share.
  • Experience with MPI and GPU jobs across several nodes.
  • Scripting in Bash or Python.
  • Clear documentation for users.

Nice to have

  • Slurm on Kubernetes.
  • Apptainer, Enroot or Pyxis for containers in Slurm jobs.
  • Lustre, BeeGFS or another parallel file system.
  • Spack or EasyBuild for software builds.

Pay and benefits

$99,000 to $159,000 a year

Full-time W-2 employment with health insurance, a retirement plan and paid leave.

No application deadline. We accept applications for this role on an ongoing basis.

Location

Remote, United States

Apply

We review applications as each team grows and contact you when there is a match.

Rillor is an equal opportunity employer. We consider qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status or any other status protected by law.

If you need an accommodation to apply, email careers@rillor.com.

Notices

Trademarks. NVIDIA, AMD, Ceph, Kubernetes and related product names are trademarks of their owners and are used only to identify products. No affiliation or endorsement is implied.