GPU Infrastructure Engineer

Posted 2 days ago

This is a fully remote position, open to applicants in Canada.

📋 Description

• Expand PolarGrid's inference network from a limited number of sites to a widely distributed array of GPU clusters.

• Take ownership of the onboarding process for new sites and nodes, overseeing everything from provisioning and burn-in to validation and phased production rollout.

• Determine when a site is ready to handle customer traffic and ensure new locations integrate into the network without compromising latency or reliability.

• Operate the global routing layer that directs requests to the most optimal serving location.

• Utilize network latency, capacity, and health indicators to adjust routing in response to live traffic.

• Establish and monitor availability objectives throughout the network.

• Develop health checks and automation systems to identify and reroute around malfunctioning hardware and models.

• Ensure rapid recovery and consistently test failure management protocols.

• Create observability, release, and network systems to efficiently manage a large-scale distributed GPU fleet.

• Implement comprehensive end-to-end tracing and latency alerting mechanisms.

• Oversee safe and controlled staged rollouts.

• Maintain network configuration as code.

• Enforce security measures and DDoS protection at the network's edge.


⛳️ Requirements

• Proven experience operating distributed systems across multiple locations or regions with high availability requirements.

• Strong understanding of networking principles: BGP, DNS, TCP/TLS, and load balancing.

• Practical experience managing bare-metal GPU fleets, including provisioning, burn-in, firmware updates, and node lifecycle management.

• Familiarity with InfiniBand or RDMA networking.

• Experience with LLM inference engines such as vLLM, SGLang, or TensorRT-LLM, focusing on multi-node serving, cold starts, and TTFT.

• Advanced observability skills, particularly in monitoring p95/p99 latency and gray failures, beyond mere uptime metrics.

• Background in designing failover strategies, graceful degradation, and automated recovery processes.

• Strong automation and infrastructure-as-code perspective, including CI/CD practices for infrastructure.

• Capability to plan capacity amidst constrained and uneven hardware availability.

• Experience in developing internal tools or control planes, preferably using Go or Rust.

• Knowledge of DDoS mitigation techniques.

• Experience managing DNS infrastructure at scale.

• Understanding of hardware security measures: TPM, measured boot, or confidential computing.

• Experience with real-time streaming inference, particularly in voice and audio applications.

• Ability to operate remote datacenter sites with on-site technicians.

• Experience with certificate automation across multiple domains.

• Proficiency in deterministic simulation testing.


🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and career advancement.

• Flexible work arrangements and a supportive work environment.

People also viewed

Qompyl8 hours ago

Cloud Infrastructure Engineer

US flagUnited States OnlyPart-timeInfrastructure Engineer
ApplyView job
Curtiss-Wright Corporation8 hours ago

Infrastructure Architect

US flagUnited States OnlyFull-timeInfrastructure Engineer$109k – $145.2k/year
ApplyView job
GT9 hours ago

Staff Infrastructure Engineer

BR flagBrazil OnlyFreelanceInfrastructure Engineer
ApplyView job
Health Care Service Corporation11 hours ago

Senior Infrastructure Engineer

US flagUnited States OnlyFull-timeInfrastructure Engineer$84.4k – $152.3k/year
ApplyView job
DPWAY S.r.l.21 hours ago

Senior Cloud and Infrastructure Architect – Migration Lead

IT flagItaly OnlyFreelanceInfrastructure Engineer
ApplyView job
RealManage22 hours ago

Principal Infrastructure Engineer

US flagUnited States OnlyFull-timeInfrastructure Engineer$180k – $210k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers