ML Infrastructure Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Collaborate closely with hardware and development teams to profile and analyze GPU performance at both the system and kernel levels.

• Assess and compare GPU performance across various platforms, architectures, and software stacks (e.g., CUDA, ROCm).

• Troubleshoot and optimize machine learning workloads for efficient execution on GPU hardware, pinpointing and addressing performance bottlenecks.

• Conduct acceptance testing for new GPU clusters to ensure that both hardware and software meet the necessary performance, stability, and compatibility standards for AI workloads.

• Execute experiments across a range of GPU system configurations to evaluate the effects of different interconnect strategies and system-level optimizations on performance and scalability.

• Create tools and dashboards to visualize performance metrics, identify bottlenecks, and track trends.

• Contribute to the development of internal tools, frameworks, and best practices.

⛳️ Requirements

• A comprehensive understanding of the theoretical foundations of machine learning.

• In-depth knowledge of performance considerations for training and inference of large neural networks (including data/tensor/context/expert parallelism, offloading, custom kernels, hardware features, attention optimizations, dynamic batching, etc.).

• Extensive experience with contemporary deep learning frameworks (such as PyTorch, JAX, Megatron-LM, Tensor-LLM).

• Solid understanding of the GPU stack: CUDA, NCCL, drivers, and associated libraries.

• Familiarity with containerized environments (e.g., Docker, Kubernetes).

• Excellent communication skills and the ability to work autonomously.

🏝️ Benefits

• Competitive compensation.

• Opportunities for career advancement and professional development.

• Flexibility and a healthy work-life balance.

• A collaborative and innovative company culture.

• The chance to contribute to impactful AI projects.

• An international environment with talented teams.