Senior Infrastructure Engineer – GPU Compute

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and manage the computational infrastructure that supports AI inference tasks.

• Oversee a diverse, multi-region GPU fleet that includes both consumer and datacenter hardware, such as the RTX 5090.

• Utilize SkyPilot, Kubernetes/k3s, and various cloud and on-premises providers to coordinate workloads.

• Establish dependable scheduling frameworks for inference tasks across the fleet.

• Enhance GPU utilization across inference workloads.

• Take responsibility for workload distribution across spot instances, on-premises resources, and cloud capacities.

• Ensure the continuous operation of the inference platform remains efficient and cost-effective.

• Optimize GPU performance through techniques like PCIe P2P, ReBAR, NUMA topology, CUDA/driver adjustments, memory settings, and network architecture.

• Create secure access for the fleet using Tailscale and Teleport.

• Build systems for observability and alerts.

• Implement zero-downtime deployments across a distributed fleet of nodes.

• Lower the cost per GPU-hour by managing spot instances, optimizing workload placement, and scheduling resources.

⛳️ Requirements

• Over 5 years of experience in infrastructure/DevOps managing large-scale production systems.

• Extensive knowledge of Kubernetes, Docker, and large-scale container orchestration.

• Strong skills in Linux systems administration.

• Proficiency in infrastructure-as-code tools like Terraform, Ansible, and Pulumi.

• Proven experience managing mission-critical, high-throughput systems.

• Solid background in infrastructure-as-code in diverse environments.

• Competence in at least one scripting or programming language, such as Python, Bash, TypeScript, or Go.

• Ability to navigate uncertainty with a strong inclination toward proactive action.

• Applicants must include a public GitHub profile in their submission.

• The GitHub profile must show at least one year of activity/history.

• Applications lacking a GitHub profile or with insufficient activity will be disregarded.

• Experience in GPU computing infrastructures, encompassing CUDA, bare-metal optimization, and kernel tuning.

• Background in operating ML training or large-scale distributed computing infrastructures.

• Familiarity with GPU fleet orchestration tools like SkyPilot, Ray, or Slurm.

• Knowledge of fleet access and networking tools, including Tailscale and Teleport.

• Understanding of network optimization and topology design.

• Experience with multi-region, globally distributed systems.

• Proficiency in Rust or low-level systems programming.

• Experience in managing on-premises data center operations.

🏝️ Benefits

• Equity allocation.

• Health, dental, and vision insurance for U.S. employees; regionally adjusted for global teams.

• Flexible Paid Time Off (PTO).

• Budget for professional development.

• Budget for conference travel.

• Remote-first work environment.

• Regular off-site team events.

• High-trust and high-velocity team culture.