Senior Applied ML Engineer – Model Router

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the routing model from start to finish, encompassing features, objectives, training, calibration, and release schedules.

• Create evaluations that match production standards, conduct router-vs-router comparisons, manage task-family holdouts, and perform analyses with error bars on pass rates.

• Transform production results, benchmark rollouts, and override feedback into actionable training signals.

• Classify failures into infrastructure and task-related issues to ensure that flawed labels do not affect the policy.

• Analyze task success, token costs, wall-clock speed, concurrency, and the economics of cached inputs.

• Execute shadow deployments, A/B tests, complete rollouts, drift monitoring, and retraining processes.

• Develop well-calibrated decision-making, implement abstention under conditions of uncertainty, and design speed-aware routing solutions.

• Collaborate with platform engineers on instrumentation, features and labels per request, as well as data paths for experiments.

⛳️ Requirements

• Over 5 years of experience in applied machine learning or machine learning research.

• Proven experience in training, evaluating, and deploying models that facilitate real-time decision-making, including ranking, recommendation, bandits, pricing, or LLM routing.

• Strong proficiency in Python and SQL at scale, capable of managing millions of request-level records.

• Ability to differentiate between model enhancements and simpler evaluations.

• Familiarity with LLM benchmarks and agentic harnesses.

• Knowledge of TTFT, decode speed, batching, prefix caching, and concurrency.

• Experience in system instrumentation and utilizing error bars.

• Proficient in Python, PyTorch, SQL at scale (ClickHouse), OpenAI-compatible and Anthropic-compatible LLM APIs, benchmark and rollout harnesses, along with GitLab CI.

• A background check may be conducted at the final stage of the hiring process.

• Must be eligible to work without visa sponsorship or a work permit provided by Cast AI.

🏝️ Benefits

• Competitive salary based on experience level.

• A flexible, remote-first global work environment.

• Equity options available.

• 10% of work time allocated for personal projects or self-improvement initiatives.

• Learning budget for both professional and personal development, including opportunities to attend international conferences and courses.

• Annual hackathon event.

• Budget for team-building activities and company events.

• Equipment budget to support your work needs.

• Additional days off available.

• Quick feedback loops facilitated by a fast-paced workflow.