Member of Technical Staff, ML Workload Platform

This is a fully remote position, open to applicants in Canada, +1 more country.

📋 Description

• Take charge of the platform responsible for scheduling and executing training, evaluation, and reinforcement-learning tasks on Kubernetes.

• Oversee capacity distribution, job failures, and platform-related visibility for researchers.

• Develop networking and runtime isolation, which includes agent and sandbox environments.

• Collaborate with the data infrastructure team to manage checkpoint and training-artifact storage.

• Diagnose production failures and address performance issues.

• Create automation, infrastructure-as-code, and observability solutions for enhanced maintainability.

• Partner with AI researchers and ML engineers to ensure stability, scalability, and observability.

• Engage in a compensated 24x7 on-call rotation.

⛳️ Requirements

• Proven experience operating a job platform, batch system, or Kubernetes controllers in a production environment.

• Familiarity with scheduling, queues, and ensuring multi-tenant fairness.

• Strong understanding of distributed systems, including retries, partial failures, and identifying what should not be automated.

• Experience with machine learning training workflows, reinforcement learning sandboxes, container networking (CNI), or large-scale checkpoint storage.

• Direct experience with GPU clusters is advantageous.

• Proficiency in programming using Python or Go.

• Experience collaborating with researchers, ML engineers, or teams with varying objectives.

• Track record of identifying bottlenecks, suggesting solutions, and implementing fixes in a fast-paced environment.

• Willingness to participate in a compensated 24x7 on-call rotation.

• Authorized to work in the country of residence.

• Must be prepared to address questions regarding Kubernetes at scale and ML/HPC infrastructure, including GPU/TPU clusters, JAX, PyTorch, TensorFlow, and high-performance computing settings.

🏝️ Benefits

• Weekly lunch stipend of $75/£75 or the equivalent in local currency.

• Comprehensive health and dental coverage.

• Dedicated budget for mental health support.

• RRSP matching program.

• 401K plan.

• Pension scheme.

• 100% parental leave top-up for up to 6 months, applicable to either parent.

• Annual enrichment allowance for arts and culture, fitness/wellness, quality time, and workspace enhancements.

• Education and learning stipend for conferences, courses, and coaching opportunities.

• 6 weeks of paid vacation (30 working days).

• Travel budget for visiting other offices if working remotely.

• Annual company offsite event.

• Daily lunch program, snacks, and frequent community and social activities for in-office employees.

• Co-working benefit available for employees not located near an office.

• $500 stipend for home office setup.

• Compensated 24x7 on-call rotation.