Strategic Projects Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Create comprehensive data collection and evaluation pipelines for RLVR, RLHF, SFT, red-teaming, and model evaluation processes.

• Develop, test, and refine AI agents that automate tasks within the pipeline — including quality gate reviews, expert matching, output flagging, and detecting throughput anomalies.

• Establish data quality benchmarks across annotation, evaluation, and expert review outputs.

• Collaborate directly with AI researchers, TPMs, and PMs within client organizations.

• Keep up-to-date with advancements in LLM post-training, evaluation techniques, and data tools.

⛳️ Requirements

• Proficiency in Python and SQL for data manipulation, monitoring pipelines, and conducting quality analysis.

• Familiarity with the internals of LLMs: RLHF/SFT training cycles, the impact of prompt structure on output distribution, and qualities of RL environment setups (tool usage) for data collection/evaluation projects.

• Practical experience with at least one agentic or LLM workflow framework (e.g., LangChain, DSPy, AutoGen, or direct tool usage via API).

• Proven track record of managing a data or ML pipeline from inception to delivery — including quality design beyond just tracking throughput.

• Excellent written communication skills: you will produce technical guidelines and rubrics for distributed expert workers to follow accurately, and you will communicate pipeline performance to senior researchers.

• Ability to navigate ambiguity in a dynamic environment where model requirements and client priorities may change rapidly.

🏝️ Benefits

• Offers Equity