Data Engineering Lead – Data Quality Systems

This is a fully remote position, open to applicants in India.

📋 Description

• Supervise a compact team of 3–5 Applied AI Engineers.

• Design and construct verification, sampling, and scoring pipelines utilizing billions of company and individual records across 13 markets.

• Develop frameworks for quality checks, harnesses, abstractions, and specifications for SKILL.md.

• Create evaluation harnesses for validation and extraction based on LLM.

• Generate labeled evaluation sets and monitor precision and recall metrics.

• Calibrate judges, manage version prompts, and identify drift in vendor model updates.

• Establish pre- and post-production release gates to prevent the introduction of erroneous data.

• Develop tools for failure analysis and triage.

• Investigate quality incidents on a large scale, implement fixes, and transform failures into automated checks that are permanent.

• Provide technical leadership, carry out code reviews, collaborate with engineers, and foster the development of team members.

• Write complex production code and assist engineers in utilizing the frameworks.

⛳️ Requirements

• Over 7 years of experience in building production data systems within business-critical settings.

• Proficient in handling billions of rows of data.

• Proven experience in developing data quality systems and frameworks, which includes validation engines, anomaly detection, scoring, sampling strategies, and reconciling against ground truth.

• Skilled in designing evaluation or test harnesses that are critical for other engineers.

• Experience in technically leading a small team while maintaining a hands-on approach.

• Strong proficiency in Python and advanced SQL.

• Hands-on experience in operating LLMs with evaluation sets, versioned prompts, logged traces, cost ceilings, and precision/recall debugging.

• Familiarity with delivering real work using agentic IDEs such as Claude Code, Cursor, or similar tools.

• Excellent judgment regarding deterministic rules versus LLM-based checks.

• Highly regarded: Experience in B2B data, cloud data platforms, AWS, Airflow or similar, vector databases, embeddings, retrieval patterns, and experience in startups or scale-ups.

🏝️ Benefits

• Competitive base salary plus significant equity.

• Flexible working hours: no fixed schedule.

• Fully remote work environment.

• Commitment to being an equal opportunity employer.

• Small senior teams with streamlined processes.

• Weekly releases with a goal of moving toward daily releases.

• Teams are responsible for their entire stack end to end.