
Data Annotation Specialist, Software Engineering
Posted 11 hours ago

Posted 11 hours ago
This is a fully remote position, open to applicants in Canada.
• Analyze the model's performance in handling coding requests, workflows, and inquiries related to the code base using available tools.
• Evaluate agent trajectories and the model's capabilities for generating and debugging code.
• Engage models to tackle intricate coding tasks and assess the precision of the produced responses.
• Annotate, edit, and enhance both machine-generated and human-created outputs in software engineering.
• Document quality and performance trends associated with model/agent behavior and project assignments.
• Proven expertise in software engineering, backed by both academic study and 3-5 years of relevant professional experience.
• Strong knowledge and comprehension of Python, as well as familiarity with the following programming languages: Java, JavaScript, Go, and SQL (any dialect).
• Previous experience in designing, building, and deploying APIs.
• Prior work with code agents (such as OpenCode, Claude Code, Codex, Cursor) or assessing agent trajectories is a significant advantage.
• Performance incentives included!
EverCommerce
PlanetScale
Slingshot Aerospace
Upstart
Get handpicked remote jobs straight to your inbox weekly.