Research Engineer – RL Infrastructure

This is a fully remote position, open to applicants in California.

📋 Description

• Develop and enhance the systems infrastructure supporting large-scale reinforcement learning (RL) and distributed training processes by contributing to the prime-rl framework.

• Enhance overall training efficiency through optimization across compute, memory, networking, and scheduling layers.

• Design and execute low-level performance enhancements, including kernels, communication pathways, and runtime optimizations.

• Engage in the development of distributed training systems encompassing data, tensor, and pipeline parallel workloads.

• Play a key role in defining the architecture of the RL training stack, which includes asynchronous rollout systems and post-training frameworks.

• Contribute to both open-source libraries and internal infrastructure utilized for frontier-scale model training.

• Collaborate with researchers and infrastructure engineers to convert identified bottlenecks into tangible system improvements.

• Keep abreast of advancements in training systems, inference systems, compiler/runtime tools, and hardware-aware optimization strategies.

⛳️ Requirements

• Extensive systems engineering experience in AI/ML infrastructure, particularly focused on large-scale model training or inference.

• Profound knowledge of PyTorch and distributed training frameworks such as PyTorch Distributed, DeepSpeed, FSDP, Megatron, vLLM, Ray, or similar tools.

• Proven experience in optimizing training performance concerning kernels, memory movement, communication overhead, or parallelization strategies.

• Practical experience with data parallelism, tensor parallelism, and pipeline parallelism.

• Strong grasp of GPU architecture, profiling, and performance debugging techniques.

• Capability to identify bottlenecks throughout the stack and initiate improvements based on fundamental principles.

• Comfort in a dynamic environment characterized by ambiguous challenges and a high level of ownership.

• Experience in writing or optimizing CUDA / Triton kernels (optional).

• Background in compiler or runtime optimization for ML systems (optional).

• Experience in RL training infrastructure, rollout systems, or asynchronous training pipelines (optional).

• Familiarity with multi-node GPU clusters and high-performance networking (optional).

• Contributions to open-source ML systems or infrastructure initiatives (optional).

• Interest in publishing technical work or sharing insights via engineering blogs and technical writing (optional).

🏝️ Benefits

• Equity incentives.

• Flexible work arrangements, allowing remote work or in-person collaboration at offices in San Francisco.

• Visa sponsorship.

• Relocation assistance for international candidates.

• Quarterly team off-sites.

• Hackathons.

• Opportunities for conferences and ongoing learning.

• Chance to collaborate with a talented, driven, and mission-focused team.