Staff AI Operations Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Collaborate with department leaders to identify key workflows and assess where AI models can provide assistance.

• Establish success metrics such as cycle time, quality, cost, and risk; ensure results are measured and reported transparently.

• Implement production systems equipped with evaluation harnesses, human-in-the-loop reviews, and continuous monitoring.

• Promote adoption through user research, comprehensive documentation, and strategic implementation plans.

• Create frameworks for model access, orchestration, agents, prompt and tool management, evaluation, logging, and cost management.

• Enhance the existing infrastructure of the Data team.

• Co-manage shared components with Engineering and coordinate interdependencies in partner roadmaps.

• Develop templates, playbooks, best practices, safety checklists, and evaluation methodologies.

• Facilitate office hours, demonstrations, and community practice sessions.

• Design AI enablement resources for go-to-market strategies, operations, and leadership.

• Assist employees in building the judgment necessary for appropriate AI usage.

• Collaborate with Security, IT, and Legal on access controls, data management, vendor risk, and policy compliance.

• Apply data classification standards to AI use cases and specify necessary audit trails for sensitive data in LLM workflows.

⛳️ Requirements

• Extensive experience in deploying systems into real business workflows, managing end-to-end production processes that non-technical staff rely on daily.

• Proficient with contemporary AI technologies: LLMs, retrieval systems, tool utilization and agents, evaluation processes, and guardrails.

• Capacity to learn business functions such as support, sales, or finance sufficiently to enhance them.

• Proven ability to lead cross-functional projects without formal authority.

• Comfortable navigating the complexities of security, privacy, and compliance trade-offs, including data classification, retention, access controls, and vendor risk.

• Strong teaching instincts; adept at simplifying complex concepts and producing clear documentation.

• Proven ability to stay updated with industry changes and adjust perspectives based on new evidence.

• Proficiency in SQL and familiarity with a modern data stack (preferred).

• Experience with workflow automation or enterprise systems (preferred).

• Understanding of observability practices for AI systems (preferred).

• Change management experience within a distributed, multi-functional organization (preferred).

• Interest in external evangelism (preferred).

• Must be able to verify identity and eligibility to work.

• Honeycomb is unable to sponsor or support visa transfers at this time.

🏝️ Benefits

• Competitive equity through an employee-friendly stock program.

• Transparent compensation based on experience levels.

• Unlimited paid time off (PTO).

• A distributed-first culture and mindset.

• Stipend for home office, co-working spaces, and internet expenses.

• Comprehensive benefits coverage for employees, with additional options for dependents.

• Up to 16 weeks of paid parental leave, regardless of the path to parenthood.

• Annual development allowance.

• Reasonable accommodations provided during the hiring process.