Manager, AI Operations

This is a fully remote position, open to applicants in Tennessee.

📋 Description

• Take charge of deployment support, monitoring, and the daily reliability of traditional ML, Generative AI, and agentic AI systems in a production environment.

• Oversee incident response and conduct root-cause analysis for AI/ML services.

• Collaborate with MLOps, Cloud Engineering, and Infrastructure teams to develop and sustain deployment pipelines.

• Create observability practices that encompass drift detection, latency, uptime, accuracy decay, and agent action-level tracing.

• Establish production health metrics and direct performance degradation issues to build teams for resolution.

• Develop dashboards and alerting mechanisms for real-time insights into AI system health.

• Integrate AI systems with Security, Infrastructure, and IT teams to uphold security and access-control standards.

• Implement and manage fairness, safety, and bias-monitoring metrics as defined by AI Governance.

• Coordinate the handoff of the AI Build team into production and lead production-readiness evaluations.

• Lead and expand a production support and observability team.

• Prioritize tasks, manage team workload, mentor team members, and foster an operational on-call culture.

• Establish Ethical AI best practices and Standard Operating Procedures (SOPs) throughout the AI stack.

• Aim for annual reductions in AI development and runtime expenses.

• Collaborate with Governance and Compliance to create measurable, monitored, and auditable standards.

⛳️ Requirements

• 7+ years of experience in AI/ML operations, MLOps, or production data science, with at least 2+ years in a leadership role.

• Practical experience in operating traditional ML and GenAI systems in production settings, including deployment, monitoring, and incident response.

• Profound understanding of observability practices, including drift detection, model performance monitoring, logging, and alerting at scale.

• Experience working alongside Security and Infrastructure teams regarding production risk and access controls.

• Exceptional communication skills to convey operational risks to both technical and business stakeholders.

• Previous experience in healthcare, health tech, payer, or provider environments is preferred.

• Familiarity with operating agentic AI systems, including tool-use monitoring, guardrails, and action-level tracing, is preferred.

• Knowledge of Datadog, AWS CloudWatch, LangFuse, and native AWS cloud platform capabilities is preferred.

• Experience working in conjunction with a dedicated AI Governance function is preferred.

• Must successfully pass a background check.

• Must be authorized to work in the United States on a full-time, permanent basis without current or future visa sponsorship.

🏝️ Benefits

• Medical/Dental/Vision insurance.

• 401K company match.

• Flexible Paid Time Off.

• Paid parental leave.

• HSA/FSA options.

• Educational reimbursement program.

• Employer-paid EAP & Mental Health services.