AI Evaluator – Staff Software Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Assess end-to-end AI-generated coding interactions.

• Determine if responses, preambles, reasoning, and outputs demonstrate strong engineering judgment.

• Evaluate the quality and usefulness of explanations and reasoning, not just the accuracy of code.

• Differentiate between various levels of response quality.

• Deliver clear, opinionated feedback on what was effective, what did not work, and what seemed misleading.

• Assist in defining what exemplary interactions should look like for tools such as Cursor.

• Make subjective yet rigorous evaluations regarding the behavior of AI coding agents.

⛳️ Requirements

• Staff / Principal-level engineer or equivalent experience required.

• Strong expertise in Go, Python, Rust, or TypeScript / JavaScript.

• Practical experience with OpenAI Codex, Claude Code, and Cursor.

• Deep understanding of contemporary AI-assisted development workflows.

• Capability to assess code without the need to execute or thoroughly review every line.

• Comfortable providing direct, opinionated feedback.

• High standards regarding engineering quality.

• Proficiency in English at C1 level or higher, or native/first-language English speaker.

• Submission of resume, LinkedIn profile, and successful completion of a take-home evaluation exercise.

• Background check required.

🏝️ Benefits

• Short-term contract engagement.

• Flexible commitment of approximately 20–40 hours per week.

• Engagement duration ranging from 2 weeks to 6 months.