Applied AI Operations Lead

This is a fully remote position, open to applicants in California.

📋 Description

• Deploy and set up AI agents for clinical documentation, scheduling, billing, communications, and workflow automation.

• Perform workflow evaluations and design agent implementations that yield measurable ROI.

• Configure and enhance agent performance through prompt engineering, context management, and tool selection.

• Develop and maintain customer-specific evaluation frameworks.

• Uphold AI governance frameworks to ensure safe and reliable production operations.

• Design and implement monitoring systems for accuracy tracking, drift detection, and quality assurance.

• Collaborate with clinical and compliance teams to meet regulatory and operational standards.

• Create and deliver training programs for care teams.

• Work with OpenAI, Claude, Gemini, and the Canvas SDK to troubleshoot and refine agent behavior.

• Partner with Applied AI engineering to provide insights for product enhancements.

• Establish best practices, implementation templates, and governance playbooks.

• Advise Canvas customers on AI-driven healthcare operations and strategy.

• Lead discussions with executives regarding AI strategy and competitive advantages.

• Develop customer success stories, performance metrics, and ROI analyses.

• Collaborate with sales teams on strategic initiatives.

• Work alongside engineering, customer success, and technical marketing teams.

⛳️ Requirements

• A minimum of 5 years in technical customer-facing roles such as solutions engineering, technical consulting, or customer success engineering.

• Proven experience in deploying and managing LLM-based applications in production settings.

• Demonstrated success in driving enterprise technology implementations that result in measurable business outcomes.

• Strong project management capabilities with the capacity to manage multiple customer engagements simultaneously.

• Hands-on experience with products from foundation model providers (OpenAI, Anthropic, Google) and AI tools for low-code automation, along with an understanding of their features and limitations.

• Knowledge of AI safety, alignment, and governance principles.

• Experience with evaluation frameworks, A/B testing, and performance monitoring for AI systems.

• Practical understanding of the boundaries and limitations of AI products, along with an awareness of the iterative process needed to achieve high-quality results.

• Required healthcare technology experience, with the ability to provide workflow guidance and develop automations tailored to healthcare users' needs.

• Familiarity with HIPAA, clinical documentation standards, and healthcare compliance considerations.

• Ability to rapidly acquire domain expertise and engage credibly with clinical stakeholders.

• Comfort with ambiguity and the ability to impose structure in undefined scenarios.

• Outstanding written and verbal communication skills, with a knack for creating engaging technical content.

• Strong analytical and problem-solving skills, employing a data-driven approach to decision-making.

• Experience in creating and delivering technical training to diverse audiences.

• Over 5 years of experience in implementing workflow tools and automation solutions in enterprise environments, including at least 2 years of hands-on experience with LLM-based systems.

• In-depth familiarity with foundation model capabilities and practical experience in context engineering and adversarial testing.

• Ability to translate technical complexity into business value and communicate effectively with technical teams and C-suite executives.

• Consultative approach to customer engagement, diagnosing issues, designing solutions, and driving change management.

• Experience in healthcare technology or a strong willingness to quickly develop domain knowledge in clinical and operational workflows.

• Technical proficiency combined with exceptional communication skills, including documentation, training, and stakeholder influence.

🏝️ Benefits

• Competitive Salary & Equity Package.

• Health Insurance (includes an option for 100% family coverage).

• 401k.

• Paid Maternity/Paternity Leave (12 weeks).

• Flexible/unlimited PTO.

• Home Office Stipend.