Senior Manager, AI Engineering

This is a fully remote position, open to applicants in Arizona, +10 more states.

📋 Description

• Take ownership of the enterprise AI platform, which encompasses governed access to foundational models, a unified gateway, model routing, prompt and context management, and cost control mechanisms.

• Define and uphold reusable AI engineering patterns for retrieval augmented generation, agents, orchestration, and the architecture of large language model applications.

• Lead the framework for evaluating AI systems, incorporating offline evaluation suites, online quality monitoring, regression testing, and criteria for release acceptance.

• Deploy AI applications into production for both customer-facing and internal use cases.

• Oversee operational quality for production AI systems, which includes monitoring latency, availability, cost per interaction, detecting quality regressions, and ensuring support coverage.

• Monitor the landscape of foundational models and AI tools, managing transitions as models and providers evolve.

• Own the AI engineering roadmap and staffing strategy.

• Manage and mentor AI engineers while coordinating with external AI delivery partners.

• Set production-readiness standards that include guardrails, safety controls, human-in-the-loop design, fallback procedures, and incident response protocols.

• Enforce responsible AI policies as engineering controls, which consist of model risk documentation, evaluation evidence, output monitoring, and audit trails.

• Collaborate with business, digital, data, machine learning engineering, enterprise architecture, security, privacy, and infrastructure teams to prioritize AI use cases.

⛳️ Requirements

• A Bachelor’s degree is required, ideally in Computer Science, Engineering, or a related technical discipline.

• 8–12 years of experience in software, data, or machine learning engineering, specifically in building production systems.

• 3–5+ years of experience leading engineering teams within cloud-native environments.

• At least 3 years of experience in delivering production AI or machine learning systems.

• A minimum of 2 years of experience in developing generative AI or large language model applications.

• Proven experience in transitioning generative AI from proof of concept to governed production, with a focus on evaluation, guardrails, and monitoring.

• Hands-on experience with retrieval augmented generation, agent frameworks, tool usage, and orchestration methods.

• Familiarity with foundational model platforms such as AWS Bedrock, Azure OpenAI, or their equivalents.

• In-depth understanding of large language model application architecture.

• Strong knowledge of AI evaluation methods, along with safety, security, and cost considerations for AI systems.

• Excellent leadership and people-management capabilities.

• Exceptional problem-solving, analytical thinking, and decision-making abilities.

• Strong communication and influencing skills across both technical and business stakeholders.

• Comfortable working in a dynamic, matrixed, and global environment.

🏝️ Benefits

• Competitive salary and bonus opportunities.

• Financial planning and wellbeing initiatives.

• Paid time off.

• Employee discounts.

• Community engagement programs.

• Opportunities for personal and professional development.

• Health and wellness initiatives.