Senior Machine Learning Engineering Manager – AI Quality and Governance

This is a fully remote position, open to applicants in United States.

📋 Description

• Lead, mentor, and cultivate a multidisciplinary team of software, machine learning, and quality engineers.

• Foster a culture centered around technical excellence, ownership of quality, experimentation, and ongoing improvement.

• Set clear team priorities while effectively managing platform investments, product requirements, and enterprise risk.

• Define and implement a thorough quality strategy for Workiva's AI platform and products, covering unit, integration, end-to-end, performance, resilience, security, and production testing.

• Establish quantifiable quality standards, release-readiness benchmarks, and automated quality gates for AI and agentic features.

• Enhance testing methodologies for nondeterministic systems, including RAG pipelines, agents, prompts, models, tools, and multi-step workflows.

• Identify regressions, model or data drift, unsafe behaviors, and compromised customer experiences both pre- and post-release.

• Oversee the architecture and development of a scalable, self-service evaluation platform for generative AI, RAG, and agentic systems.

• Integrate governance within the AI lifecycle through traceability, lineage, versioning, documentation, risk classification, approval workflows, and verifiable evidence.

• Collaborate with Product, Program Management, UX, UXR, Data Science, Security, Legal, Risk, and engineering leaders to outline quality expectations and development roadmaps.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field (or equivalent experience).

• 10+ years of experience in software engineering, ML engineering, quality engineering, or similar roles, including over 4 years in a leadership position.

• Strong foundational knowledge in software engineering and systems design, with a proven track record of delivering and operating production SaaS or platform solutions.

• Proven experience in establishing automated quality practices for distributed, cloud-based products.

• Practical understanding of the generative AI development lifecycle and the complexities involved in evaluating nondeterministic systems.

• Familiarity with generative AI concepts: LLMs, RAG, embeddings, vector/hybrid search, agents, tool utilization, and prompt orchestration.

• Experience in defining measurable quality criteria through data, experimentation, telemetry, and production signals.

• Knowledgeable in cloud-native architectures on AWS, Azure, or GCP.

🏝️ Benefits

• A discretionary bonus typically awarded annually.

• Restricted Stock Units offered at the time of hire.

• 401(k) matching and a comprehensive benefits package for employees.