AI Interaction Evaluator – Codex, Claude Code

This is a fully remote position, open to applicants in United States, +13 more states.

📋 Description

• Assess the complete cycle of AI-generated coding interactions.

• Evaluate the usefulness and accuracy of outputs at a high level, ensuring they align with the thought process of a competent engineer.

• Analyze the quality of explanations and reasoning provided, not limited to just the code itself.

• Differentiate between various levels of response quality.

• Deliver clear and opinionated feedback regarding what was effective, what fell short, and what seemed inaccurate or misleading.

• Contribute to defining the standards of excellence in interactions with tools like Cursor.

• Determine if the responses, preambles, reasoning, and outputs from AI coding agents reflect strong engineering judgment.

• Engage in a take-home evaluation task and a behavioral interview.

⛳️ Requirements

• Highly experienced software engineer at the Senior+ level.

• Staff- or Principal-level engineer, or possess equivalent experience.

• Strong expertise in TypeScript/JavaScript or Python.

• Practical experience with OpenAI Codex, Claude Code, and Cursor.

• Profound understanding of contemporary AI-assisted development workflows.

• Capability to assess code without the need to execute or meticulously review every line.

• Comfortable providing direct and opinionated feedback.

• Maintain high standards for engineering quality.

• Able to make subjective yet rigorous judgments.

🏝️ Benefits

• Potential for extension beyond early May.

• Flexible part-time commitment of roughly 10–20 hours per week.