Member of Technical Staff, Inference

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Expand the capabilities of LLM and diffusion model serving.

• Operate at the heart of vLLM to enhance model execution on various hardware and architectures.

• Innovate inference runtimes for mixture-of-experts, multimodal, and agentic frameworks.

• Apply inference methodologies and model designs derived from academic research.

• Contribute to high-performance and maintainable codebases.

• Troubleshoot intricate machine-learning codebases.

• Enhance AI inference processes by making them more cost-effective and efficient.

⛳️ Requirements

• Bachelor's degree or equivalent experience in computer science, engineering, or a related field.

• Profound knowledge of transformer architectures and their variations.

• Strong programming abilities in Python, with familiarity in PyTorch internals.

• Experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or TGI.

• Capability to interpret and implement model architectures and inference techniques from research literature.

• Ability to produce high-performance and maintainable code while debugging in complex machine learning codebases.

• Preferred: in-depth understanding of KV-cache memory management, prefix caching, and hybrid model serving.

• Preferred: knowledge of reinforcement learning frameworks and algorithms applicable to LLMs.

• Preferred: experience with multimodal inference involving audio, image, video, and text.

• Contributions to open-source machine learning or systems infrastructure projects are advantageous.

• Bonus: implementation of core features in vLLM or other inference engine initiatives.

• Bonus: contributions to vLLM integrations such as verl, OpenRLHF, Unsloth, or LlamaFactory.

• Bonus: widely-shared technical articles or side projects related to vLLM or LLM inference.

• Required application materials include a resume, GitHub handle, and a link to a relevant personal project, open-source contribution, or technical blog post.

🏝️ Benefits

• Competitive benefits tailored to your location, including health coverage where applicable.

• Equity opportunities.

• Visa sponsorship available on a case-by-case basis.

• Fully remote working environment.

• Flexible scheduling with regular overlap with Pacific Time for essential meetings.