Senior Technical Support Engineer – Slurm

atNVIDIARemoteUS flagUnited StatesFull-timeSupport EngineerMid-levelSenior$105k – $247k/year

Posted 1 day ago

This is a fully remote position, open to applicants in United States.

📋 Description

• Take charge of Slurm support cases from the initial investigation to resolution for clients utilizing production AI and HPC clusters.

• Diagnose intricate issues involving slurmctld, slurmd, slurmdbd, job scheduling, node management, resource allocation, accounting, authentication, and high availability.

• Resolve Slurm configuration and policy features, which include partitions, reservations, priorities, fair-share, quality of service, backfill, preemption, GRES/TRES, cgroups, and job constraints.

• Examine performance, reliability, and scalability concerns by utilizing logs, diagnostic data, configuration analysis, reproductions, and source-level debugging when necessary.

• Identify issues across Slurm and its dependencies, such as Linux, MUNGE, databases, networking, parallel storage, containers, GPUs, and cluster-management systems.

• Provide guidance to customers on Slurm configuration, upgrades, operational practices, system resource management, and safe recovery from production incidents.

• Work in collaboration with engineering teams by creating clear technical descriptions, reproducible test cases, and well-documented defect reports.

• Produce guides, knowledge-base articles, diagnostic tools, and internal training to enhance Slurm expertise within the support organization.


⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, or a related discipline, or equivalent experience.

• Over 5 years of practical experience in administering and supporting Slurm in production HPC or AI settings, including handling business-critical outage incidents.

• Expert-level comprehension of Slurm architecture, daemons, configuration, scheduling behavior, accounting, resource management, and failure modes.

• Ability to independently identify complex Slurm incidents and guide them to a technically sound resolution.

• Extensive Linux system administration and troubleshooting experience, including systemd, cgroups, authentication, networking, and database-backed services.

• Proficiency in operating Slurm across multi-user clusters with intricate scheduling policies and diverse compute resources.

• Strong analytical and research capabilities, demonstrating proficiency in differentiating Slurm defects from configuration, integration, infrastructure, and workload issues.

• Exceptional written and verbal communication skills, including the ability to translate detailed technical findings into clear explanations and actionable recommendations.

• Experience in supporting large-scale Slurm environments comprising thousands of nodes or GPUs.

• Proven experience in diagnosing scheduler performance, job throughput, controller load, and database scaling issues.

• Familiarity with Slurm source code, plugins, SPANK, Lua job-submit plugins, or upstream issue investigation.

• Experience with containers and HPC integration technologies such as Pyxis, Enroot, Apptainer, or Singularity.

• Previous experience integrating Slurm with NVIDIA Base Command Manager, Bright Cluster Manager, or another cluster-management platform.


🏝️ Benefits

• Highly competitive salaries.

• Comprehensive benefits package.

• Equity.

• Benefits for you and your family.

People also viewed

Spherica1 day ago

Apprentice EUC Technical Support Engineer

GB flagUnited Kingdom OnlyFreelanceSupport Engineer£15.6k/year
ApplyView job
Akamai Technologies1 day ago

Systems Support Engineer

CR flagCosta Rica OnlyFull-timeSupport Engineer
ApplyView job
Cohere Health1 day ago

Integration Support Analyst

US flagUnited States OnlyFull-timeSupport Engineer$86k – $96k/year
ApplyView job
Twilio1 day ago

Technical Support Engineer 2

CO flagColombia OnlyFull-timeSupport Engineer
ApplyView job
PayAngel1 day ago

TechOps Support Engineer

IN flagIndia OnlyFull-timeSupport Engineer
ApplyView job
MeridianLink1 day ago

Integration Support Engineering Specialist

US flagUnited States OnlyFull-timeSupport Engineer$51.8k – $69k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers