Senior Software Engineer, AI Training

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Evaluate the quality of interactions with contemporary coding agents such as OpenAI Codex and Claude Code.

• Perform end-to-end assessments of AI-generated coding interactions.

• Determine if outputs are useful, accurate at a high level, and reflect strong engineering judgment.

• Analyze explanations and reasoning, not solely the code itself.

• Differentiate between various levels of response quality.

• Provide clear, opinionated written feedback on what was effective, what was lacking, and what seemed misleading.

• Assist in defining what exemplary interaction looks like with Cursor, Codex, and Claude Code.

• Make subjective yet thorough evaluations of AI coding agent behavior.

⛳️ Requirements

• Senior, Staff, or Principal-level engineer (or equivalent experience).

• Strong proficiency in TypeScript/JavaScript or Python.

• Practical experience with at least one of OpenAI Codex, Claude Code, or Cursor.

• Extensive knowledge of modern AI-assisted development workflows.

• Capable of evaluating code without executing it or examining every line.

• Strong written and spoken English (B2 or higher); necessary for providing detailed written feedback and short video explanations.

• Comfortable delivering direct, opinionated feedback.

• High standards for engineering quality.

• Prior experience in prompt design or evaluation workflows (nice to have).

• Experience mentoring senior engineers or establishing engineering standards (nice to have).

• Must complete a take-home evaluation exercise accompanied by a recorded Loom walkthrough.

• Must consent to a straightforward background check for the project.

🏝️ Benefits

• Flexible hours, ranging from 10–20 hours per week.

• Ongoing project opportunities for evaluators who perform well.

• Start as soon as the take-home assignment is completed and a project has an available position.