Senior Software Engineer – Java, AI Training

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Assess the quality of interactions with contemporary coding agents like OpenAI Codex and Claude Code.

• Review AI-generated coding interactions comprehensively.

• Evaluate whether outputs are beneficial, correct at a high level, and aligned with sound engineering judgment.

• Analyze the quality of explanations and reasoning, not solely the code itself.

• Differentiate between various levels of response quality.

• Provide concise, opinionated written feedback on aspects that were effective, ineffective, and potentially misleading.

• Assist in defining the benchmarks of excellence when collaborating with Cursor, Codex, and Claude Code.

• Create brief video explanations to articulate evaluation judgments.

⛳️ Requirements

• Senior, Staff, or Principal-level engineer (or similar experience).

• Strong expertise in TypeScript/JavaScript or Python.

• Practical experience with at least one of OpenAI Codex, Claude Code, or Cursor.

• In-depth knowledge of modern AI-assisted development workflows.

• Capability to evaluate code without the need to execute it or examine every line.

• Proficient in written and spoken English (B2 or above); essential for providing detailed written feedback and short video explanations.

• Comfortable delivering direct, opinionated feedback.

• High expectations for engineering quality.

• Previous experience with prompt design or evaluation workflows (preferred).

• Experience mentoring senior engineers or establishing engineering standards (preferred).

• Required to complete one take-home evaluation exercise with a recorded Loom walkthrough.

• Must consent to a straightforward background check for the project.

🏝️ Benefits

• Flexible work schedule of 10–20 hours per week.

• Continuous project opportunities for evaluators who excel.

• Chance to commence work as soon as the take-home evaluation is successfully completed and a project has an available position.