Applied Computer Science Benchmark Specialist

This is a fully remote position, open to applicants in United States.

📋 Description

• Create original and thought-provoking multiple-choice questions in various computer science fields.

• Evaluate the difficulty level of questions and submit them for review after authorship.

• Examine existing questions for technical precision, clarity, completeness, rigor, and solvability.

• Modify questions as needed and document the justification for the proposed changes.

• Provide one accurate answer along with nine plausible yet subtly incorrect alternatives.

• Craft structured solution explanations that illustrate the reasoning and technical principles underlying the correct answers.

• Include 1–5 authoritative references for each question.

• Implement consistent standards to assess questions and solutions, ensuring benchmark quality and reproducibility.

• Contribute to the development of gold-standard benchmarks for assessing advanced AI systems.

⛳️ Requirements

• Expertise in one or more of the following areas: Accelerator/GPU Kernel Engineering, Formal Methods & Automated Reasoning, Computer Architecture & Accelerators, Distributed Systems, DevOps & Site Reliability Engineering, Data Engineering & Databases, Cloud Computing & Infrastructure, Operating Systems & Systems Kernel, Machine Learning Engineering, Web & API Development, Embedded Systems Engineering, Computer Graphics & Game Development, or Mobile Engineering.

• A PhD or being a doctoral candidate in Computer Science, Electrical Engineering, Computer Engineering, or a closely related field.

• A Master's degree may be accepted for candidates demonstrating exceptional expertise in a specialized computer science domain.

• Strong understanding of graduate-level computer science theory, algorithms, systems, software engineering, architecture, and/or machine learning.

• Proven expertise in one or more specified technical domains.

• Excellent command of written English with the ability to convey complex technical concepts clearly, accurately, and concisely.

• Strong attention to detail and the capability to identify technically valid solutions versus plausible but incorrect alternatives.

• Expected commitment of over 10 hours per week.

• Must be eligible to work as an independent contractor; candidates on H-1B or STEM OPT are not eligible.

• Selected candidates may be required to complete a brief technical assessment or provide further details regarding their expertise.

🏝️ Benefits

• Flexible scheduling tailored to project needs.

• Fully remote and asynchronous working environment.

• Opportunity to contribute to high-quality benchmarks for the evaluation of advanced AI systems.

• Strong contributors may be considered for additional review, evaluation, or subject-matter expert roles.

• Work can be completed according to your own schedule.

• Weekly payments through Stripe or Wise, based on services provided.

• Reasonable accommodations will be offered throughout the application and engagement process.