Staff Software Engineer, ML Training Infrastructure

This is a fully remote position, open to applicants in Pennsylvania.

📋 Description

• Design and enhance components of a high-performance training platform, focusing on orchestration, training abstractions, control plane, observability, and performance optimization.

• Implement comprehensive ML model pipelines that encompass log processing, feature extraction, dataset schema design and storage, model configuration management, model training, and profiling/acceleration workflows.

• Assess the performance of training infrastructure to pinpoint and rectify performance bottlenecks.

• Advocate for system abstractions and tools that empower Machine Learning Engineers (MLEs) to swiftly iterate on their models.

• Integrate open-source tools that allow ML engineers to autonomously profile and enhance their workflows.

• Uphold high standards for engineering excellence and foster a culture of innovation within the team.

• Lead the design and development of a robust multi-tenant AI training platform.

• Collaborate with teams across ML Platform, Infrastructure, Autonomy, and Safety Evaluation.

⛳️ Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related discipline.

• Over 6 years of experience working with ML Platforms and developing ML-based applications.

• Proficient programming skills in Python, C++, or similar languages.

• Prior experience with technologies such as Lance, PyTorch, Ray Data, or their equivalents.

• Demonstrated success in building scalable and reliable infrastructure in a dynamic environment while collaborating with MLEs across various modeling teams.

• In-depth understanding of design trade-offs and the ability to articulate these trade-offs to foster alignment among cross-functional teams.

• Experience with model training, model optimization, or large-scale data processing pipelines.

• Strong analytical and problem-solving capabilities.

• Excellent verbal and written communication skills, capable of conveying complex technical concepts to non-technical audiences.

• The position may necessitate verification of residency, U.S. person status, and/or citizenship status due to U.S. national security and export control regulations.

🏝️ Benefits

• An equal opportunity workplace dedicated to inclusion, entrepreneurship, and innovation across gender, race, age, sexual orientation, religion, disability, and identity.