Senior Software Developer, LLM Infrastructure

This is a fully remote position, open to applicants in Canada.

📋 Description

• Develop model-routing architecture that incorporates LiteLLM integration to efficiently channel requests between internal and external models.

• Provision and oversee high-performance GPU infrastructure dedicated to serving open-source LLMs.

• Create evaluation frameworks (Evals) to benchmark model quality against production standards.

• Lead initiatives focused on cost optimization and infrastructure efficiency while ensuring system accuracy and reliability.

• Design and implement essential backend systems and infrastructure that support AI applications.

• Collaborate with data scientists, platform engineers, and engineering leaders to expand the GenAI footprint securely and effectively.

• Assist in defining the technical roadmap for scalable LLM infrastructure across the organization.

⛳️ Requirements

• 6+ years of experience in software engineering, specifically in building production-ready distributed systems or data platform infrastructure.

• Strong expertise in Python programming.

• Proficient in using Kubernetes and Terraform.

• Demonstrated experience in designing and maintaining highly performant and observable backend systems.

• Practical experience or a strong understanding of the contemporary AI ecosystem, including serving open-source models, vLLM, Ray, model routing, or evaluation frameworks.

• Proven ability to manage technical projects from inception to completion with significant autonomy.

• Candidates residing in Canada must possess legal eligibility to work in Canada.

🏝️ Benefits

• Comprehensive health benefits and life insurance coverage.

• Long-term group savings plan with employer matching, facilitated through Wealthsimple for Business.

• 20 vacation days annually.

• 4 wellness days each year.

• Unlimited sick leave and mental health days per year.

• Opportunity to work outside of Canada for up to 90 days per year.

• Employee resource groups, including Rainbow (2SLGBTQ), Women of WS, and Black at WS.

• Equity compensation available for permanent employees.

• Inclusive hiring practices and accommodations during the interview process.