This is a fully remote position, open to applicants in North Carolina.

📋 Description

• Develop and troubleshoot code within a live, real-time voice pipeline where both latency and accuracy impact user experience.

• Create control systems for LLMs, including guardrails, budgets, watchdogs, and secure fallback mechanisms.

• Construct and manage LLM evaluation and batch-analysis pipelines.

• Take ownership of traditional ML workflows, from data handling to scheduled production inference.

• Investigate production issues from metric anomalies to their root causes, including assembling evidence when a vendor is implicated.

⛳️ Requirements

• Over 7 years of experience in building and managing production backend systems, with robust general programming skills (our primary language is Python).

• Proficient in operating distributed systems in the cloud; adept at debugging from telemetry to root causes.

• Practical production experience with LLMs or generative AI (regardless of provider or framework), along with the discernment to avoid inappropriate model usage.

• Familiarity with the entire traditional machine learning lifecycle; focused on productionization rather than publication.

• Methodical in a regulated setting: implementing small, reviewable changes and managing sensitive data carefully.