Research Engineer, Coding Evaluation – Training Data

This is a fully remote position, open to applicants in United States.

📋 Description

• Take full ownership of coding data projects, managing them from the initial scoping and pilot design stages through to execution, iteration, and scaling.

• Craft agentic training workflows and task structures that accurately reflect real-world software engineering tasks (e.g., refactoring, debugging, code reviews, large repository navigation, and tool usage).

• Develop and refine rubrics, golden sets, and reward signals that truly reflect engineering value, rather than merely checking if code compiles.

• Assess data and worker output with a keen software engineering perspective, determining what qualifies as meeting the standards for advanced training.

• Create and implement qualification processes for coding workers, incorporating hands-on evaluations of their coding skills.

• Establish or collaborate on intricate technical environments (e.g., containers, repositories, test harnesses, sandboxes, and code execution infrastructure).

• Work alongside technical teams at our client organizations to translate broad training objectives into specific projects and technical setups.

• Collaborate closely with Surge's engineering, product, and operations teams to enhance our coding data products, internal tools, and execution workflows.

⛳️ Requirements

• 3–6+ years of professional software engineering experience in designing and maintaining real systems.

• Proficient coding skills in at least one mainstream programming language and familiarity with working in production codebases.

• A strong sense of engineering quality: you value correctness, code quality, and understand the dynamics of real engineering teams.

• Capability to analyze and troubleshoot technical environments, including containers, dependencies, and automated testing setups.

• A genuine interest in managing projects from start to finish, encompassing scoping, workflow design, execution, and ongoing enhancement.

• Exceptional written and verbal communication skills, with the ability to engage effectively with senior client engineers and convert ambiguous goals into clear, actionable plans.

• A passion for AI/ML systems and an appreciation for the role of data, evaluation, and reward design in enhancing coding capabilities.

🏝️ Benefits

• Health insurance

• 401(k) matching

• Flexible work hours

• Paid time off

• Professional development opportunities