Principal Machine Learning Engineer, Inference

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Develop first-party speech models and provide LLM services for real-time, consumer-oriented applications

• Create and manage inference processes behind modular APIs designed for high-volume, real-time tasks

• Enhance model serving concerning latency, throughput, and scalability performance

• Containerize models and facilitate their transition from research environments to dependable production systems

• Engage with inference frameworks, distributed systems, Kubernetes, Ray, load balancing, and multi-GPU/multi-node setups

• Design benchmarks or prototypes to resolve ambiguous technical challenges

• Analyze and enhance code performance running on NVIDIA GPUs

• Contribute to systems programming, open-source inference engines, or produce technical documentation

• Work collaboratively with research and engineering teams to deliver stable, high-impact systems

⛳️ Requirements

• Profound knowledge of contemporary serving frameworks and methodologies like vLLM or TRT-LLM

• Practical experience with quantization, distillation, caching methodologies, continuous batching, paged attention, and speculative decoding

• Expertise in C++, CUDA, Rust, or highly optimized Python

• Capability to profile code and enhance NVIDIA GPU performance

• Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and managing thousands of simultaneous connections

• Involvement in significant systems programming projects, open-source contributions to prominent inference engines, or in-depth technical documentation

• Proven ability to transition a model from research to production deployment and ensure its reliability

• PhD in Computer Science, Physics, or Mathematics, or equivalent practical experience in developing backend or machine learning systems

• Must possess the legal right to work in the United Kingdom

• Visa sponsorship is not available for this position

🏝️ Benefits

• Equity

• Benefits

• Potential future U.S. visa and relocation assistance for moving to the San Francisco Bay Area, subject to business needs and relevant legal and work authorization requirements