Tech AI Data Delivery Manager

This is a fully remote position, open to applicants in India.

📋 Description

• Design and execute AI data initiatives

• Develop hybrid automation that integrates deterministic tools with LLM agents

• Determine which workflow stages should employ rules versus models and create transitions

• Establish and manage agents within Claude and the native enterprise agentic environment

• Configure task specifications, data and tool integrations, schedules, monitoring, and error management

• Create quality assurance workflows, guidelines, rubrics, and scorecards

• Adjust automated checks in relation to human reviewers and assess error categories

• Oversee data collection and annotation projects from start to finish, encompassing scoping, vendors, workflows, tracking, and delivery

• Develop tools for project coordinators

• Gather, clean, and analyze production and quality data

• Generate reports on throughput, rework, error rates, and contributor performance

• Conduct testing of automation prior to release and establish continuous auditing workflows

• Draft specifications, tickets, and runbooks for coordinators and engineers

⛳️ Requirements

• Proficient in JavaScript and Python at an intermediate level, adequate for building and maintaining production automation

• Practical experience in building agents in Claude, including projects, skills, connectors, and scheduled tasks

• Capability to develop agents in the native enterprise agentic environment and link APIs with native infrastructure

• Experience in LLM evaluation, including prompt and rubric design and calibration against human judgment

• Background in data analytics with experience in the data collection or annotation sector

• Experience in managing projects involving external vendors or contributors and multiple stakeholders

• Ability to make and take ownership of decisions with incomplete information

🏝️ Benefits

• Flexible remote work arrangement

• Full-time, fixed-term employment (1 year with the option to extend)