Senior Engineering Manager, AI Quality – Governance

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the AI evaluations and observability platform, which includes tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool utilization, and user outcomes.

• Design and implement both automated and human-in-the-loop evaluation strategies, incorporating regression checks for quality, safety, latency, and cost into engineering and release workflows.

• Define the standards that characterize production-ready AI across the organization, including evaluation criteria, release gates, incident response playbooks, and long-term quality metrics.

• Build and maintain AI guardrails and policy enforcement mechanisms, such as content controls, PII detection and redaction, audit logging, and policy checks at the request or workflow level.

• Convert emerging governance and risk requirements into functional engineering systems and platform controls instead of relying on static documentation.

• Manage platform-level SLOs and provide tier-two incident support for AI behavior concerns, collaborating with product teams who are the primary owners of the features they develop.

• Serve as an internal expert on AI quality and governance and engage in customer-facing discussions where product quality, safety, observability, or governance posture needs to be clearly communicated.

• Recruit, mentor, and develop the team over time; this combined role may later transition into distinct Quality/Observability and Governance/Policy teams.

⛳️ Requirements

• A minimum of 8 years of software engineering experience, with significant experience in leading engineers as a manager or technical lead.

• Extensive experience in designing and operating production backend systems and APIs.

• Proven hands-on experience in building, shipping, or operating production AI systems, ideally including LLM-powered or agentic workflows.

• Experience with AI observability, evaluation, or debugging systems, whether through platforms like LangSmith or LangFuse, or through custom-built equivalents.

• Practical experience in designing or operating AI guardrails, including content filtering, redaction, access controls, or other measures related to model or agent behavior.

• Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001, and the ability to engage thoughtfully with legal, compliance, and security stakeholders.

• A proactive approach towards hands-on execution, platform thinking, and creating streamlined processes for other teams.

🏝️ Benefits

• Health, Dental & Vision Insurance *

• 401 (k) + Employer Match *

• Unlimited PTO + 11 Paid Holidays + 4 Annual Paid Global Wellness Days Off

• STD, LTD & Group Life Insurance

• Paid Parental Leave

• Pet Insurance

• FSA & HSA Options

• Employee Assistance Program

• Remote Work

• Career Advancement & Professional Development Opportunities

• Employee Recognition

• LinkedIn Learning Platform