Software Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Create realistic, multi-step software engineering challenges that reflect practical development workflows, encompassing implementation, debugging, system configuration, testing, and documentation.

• Produce thorough reference solutions in Python, ensuring that each task is reproducible, verifiable, and includes appropriate validation checks.

• Utilize AI coding assistants within your development process while assessing their advantages, limitations, and failure modes in handling complex engineering tasks.

• Evaluate benchmark tasks generated by other experts, offering detailed feedback regarding technical accuracy, clarity, completeness, and difficulty level.

• Examine AI-generated solutions to pinpoint implementation errors, debugging issues, reasoning gaps, and opportunities for enhancing benchmark quality.

• Collaborate effectively with AI researchers and engineering teams to improve evaluation methodologies and uphold consistently high technical standards.

⛳️ Requirements

• Master's degree, PhD, or equivalent practical experience in Computer Science, Software Engineering, or another STEM field that involves significant programming and technical problem-solving.

• At least 1 year of professional experience in software engineering, research engineering, applied research, or another technically intensive engineering position.

• Strong hands-on expertise in Python, including software design, scripting, debugging, testing, and producing clean, maintainable code.

• Substantial experience with Git, integrated development environments (IDEs), and contemporary software development practices.

• Comprehensive understanding of software architecture, debugging techniques, and engineering best practices.

• Preferred experience with AI coding assistants, prompt engineering strategies, or AI agent workflows.

• Highly desirable experience in AI evaluation, benchmark development, AI training, or task creation.

• Exceptional analytical thinking, creativity, attention to detail, and the capability to independently tackle complex, open-ended technical problems.

• Strong written communication skills for documenting technical solutions and engineering decisions.

• Ability to consistently commit approximately 35 hours per week.

🏝️ Benefits

• Opportunity to work on innovative projects in a collaborative environment.

• Access to cutting-edge technology and tools in software development.

• Flexible working hours to support work-life balance.

• Continuous professional development and learning opportunities.