Senior AI Development Lead

atAirbnbRemoteUS flagUnited StatesFull-timeArtificial IntelligenceSenior$157k – $171k/year

Posted 9 hours ago

This is a fully remote position, open to applicants in United States.

πŸ“‹ Description

β€’ Lead the development of AI workflows and features from conception to deployment, establishing quality benchmarks and assessment plans in collaboration with engineering, product, and business stakeholders.

β€’ Prioritize initiatives based on business impact and negotiate trade-offs with the manager alongside business and product leaders.

β€’ Analyze current processes to identify opportunities for AI implementation and determine where human oversight is essential.

β€’ Design and construct AI workflows, which include prompts, pipelines, and tools.

β€’ Oversee quality control and monitor any shifts post-launch.

β€’ Develop evaluation frameworks, create test sets, establish accuracy and agreement metrics, and align LLM judges with human reviewers.

β€’ Collaborate with engineers on architectural and tooling decisions while rigorously assessing estimates and project scope.

β€’ Transform ambiguous guidelines into clear and consistent reviewer criteria in partnership with product, Legal, and Policy teams.

β€’ Generate labeling guidelines and human-verified labels in conjunction with subject matter experts.

β€’ Manage large-scale labeling operations with vendor teams and review outcomes using agreement metrics and spot checks.

β€’ Establish launch quality benchmarks for AI features and provide documentation along with go/no-go recommendations.

β€’ Create reusable evaluation and workflow components, document insights, and promote adoption across various teams.

β€’ Update methods in line with changes in models, tools, and policies, and advise on retiring outdated methodologies.


⛳️ Requirements

β€’ 8+ years of experience in product or program management, analytics, consulting, or applied AI.

β€’ Demonstrated success in guiding technical programs from concept to execution while managing outcomes such as backlog, cycle time, or launch velocity.

β€’ Proficient in SQL.

β€’ Strong understanding of accuracy, precision, recall, sampling, and reviewer consensus.

β€’ Experience in deploying an AI workflow, agent, or automation that is utilized by others, with evidence of measurable before-and-after results.

β€’ Technical comprehension sufficient for evaluating engineering estimates, understanding prompts, pipelines, and evaluations, and identifying workarounds.

β€’ Proven ability to quantify chaotic processes and assess business results.

β€’ Capability to evaluate qualitative feedback and resolve reviewer discrepancies.

β€’ Excellent writing skills and ability to collaborate with sales, operations, PMs, and engineers.

β€’ Openness to tackle problems head-on and experiment directly.

β€’ Must reside in a state where Airbnb, Inc. has a registered entity.

β€’ Preferred: hands-on experience with Python and APIs.

β€’ Preferred: experience in LLM evaluation.

β€’ Preferred: background in experimentation and measurement.

β€’ Preferred: experience in trust and safety, content moderation, customer support automation, annotation operations, or similar human-in-the-loop activities at scale.

β€’ Preferred: experience in a two-sided marketplace, travel, or other operations-intensive industries.


🏝️ Benefits

β€’ Eligibility for bonuses.

β€’ Eligibility for equity.

β€’ Access to benefits.

β€’ Employee travel credits.

β€’ Opportunities for occasional work at an Airbnb office or participation in offsite events, as coordinated with the manager.

β€’ Reasonable accommodations provided throughout the recruitment process.

People also viewed

CVS Health Careers8 hours ago

Senior Analyst, Clinical AI and Data Strategy

US flagTexas OnlyFull-timeArtificial Intelligence$47k – $102k/year
ApplyView job
Sourcegraph8 hours ago

AI Transformation Architect

US flagUnited States OnlyFull-timeArtificial Intelligence$200k/year
ApplyView job
MCA Connect8 hours ago

Senior Consultant, Data & AI – Microsoft

US flagUnited States OnlyFull-timeArtificial Intelligence$120k – $145k/year
ApplyView job
Lightly8 hours ago

AI Operations Monitor – Digital Operations & Incident Response

CH flagSwitzerland OnlyFreelanceArtificial Intelligence$20 – $30/hour
ApplyView job
WRS Health8 hours ago

AI & Automation Marketing Lead

US flagUnited States OnlyFreelanceArtificial Intelligence
ApplyView job
WRS Health8 hours ago

AI and Automation Marketing Lead

US flagUnited States OnlyFreelanceArtificial Intelligence
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers