Principal Engineer, Model Performance

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of inference performance from start to finish across various kernels, serving engines, model-level optimizations, and benchmarks for image, video, audio, 3D, and LLM workloads.

• Establish the model performance roadmap informed by actual traffic, costs, and upcoming hardware developments.

• Develop and refine CUDA kernels, merge attention and MLP paths, introduce FP8, FP4, and INT4 quantization, and implement parallelism and batching strategies on multi-GPU nodes.

• Create a streamlined process for transitioning newly released open models from their weights to optimized, production-ready deployments in just days.

• Select and shape the serving stack, encompassing TensorRT-LLM, vLLM, SGLang, custom diffusion and video pipelines, and the Sonic Inference Engine runtime.

• Oversee reproducible, hardware-aware benchmarking and performance regression systems integrated into CI.

• Collaborate with infrastructure and hardware engineering teams to address GPU memory, storage, and network bottlenecks.

• Build and lead the Model Performance team, which includes hiring, mentoring, and establishing technical standards.

• Leverage AI agents for automated kernel search and tuning, benchmark generation, profiling triage, and documentation.

⛳️ Requirements

• 10+ years of experience in GPU, HPC, or deep learning systems engineering, including several years focused on optimizing inference performance for production models at scale.

• Proven history of delivering performance enhancements and advancing state-of-the-art models from reference implementation to the Pareto frontier of latency and cost on actual hardware.

• Extensive, up-to-date, hands-on expertise in CUDA and GPU architecture, including kernel design, memory hierarchy and occupancy, Tensor Cores, fused kernels, CUDA Graphs, NCCL, Nsight Systems, and Nsight Compute.

• Production experience with at least one major inference engine: TensorRT-LLM, vLLM, SGLang, or similar.

• Knowledge of continuous batching, paged KV cache, speculative decoding, prefill/decode disaggregation, quantization, tensor parallelism, pipeline parallelism, and expert parallelism.

• Proficient in Python and C++.

• Familiarity with PyTorch internals, torch.compile, and Triton.

• Experience leading engineers, building or managing a performance team, hiring, setting technical direction, and staying hands-on.

• Regular use of AI for kernel search, profiling analysis, benchmarking, documentation, and code review.

• Capability to take ownership of outcomes in a dynamic, ambiguous environment, and maintain accountability for latency, cost, and quality across hundreds of thousands of models.

🏝️ Benefits

• Meaningful stock options.

• Remote-first work arrangement.

• In-person meetings twice a year.

• Core hours for collaboration and calls; flexible schedule outside of core hours.

• Generous paid time off, including vacation, sick days, and public holidays.

• Paid maternity, paternity, and caregiver leave.

• Twice-yearly company retreats.

• GPU budget and company support for inference performance initiatives.