Mercor Research Fellowship – APEX

This is a fully remote position, open to applicants in California.

📋 Description

• Propose and define a new benchmark or evaluation method in a less-explored APEX domain or a significantly more challenging version of an existing one.

• Create task specifications and grading criteria in collaboration with Mercor’s qualified domain experts.

• Develop and validate benchmarks through pilot tasks, scoring calibration, and stress testing to identify contamination and exploitable shortcuts.

• Test frontier models against benchmarks and assess failure modes.

• Publish findings as a paper, an open dataset, an APEX leaderboard, or a methodology adopted internally.

• Collaborate with Mercor’s research and engineering teams to integrate insights into APEX’s public benchmark collection.

• Work closely with the APEX research team and engage with enterprise evaluation challenges from Fortune 500 companies and frontier-lab partners.

⛳️ Requirements

• A genuine passion for evaluation as a research field.

• A background in Computer Science, Machine Learning, statistics, or a related area such as measurement, psychometrics, Human-Computer Interaction (HCI), or social sciences.

• A specific, well-defined concept for a benchmark or evaluation technique to develop.

• Comfort in a startup setting characterized by rapid iteration and less guidance than in an academic lab.

• Availability to commit at least 20 hours per week throughout the fellowship duration.

• A minimum commitment of 30 hours per week is expected; full-time engagement is preferred.

• Bonus: experience in agentic evaluation, reinforcement learning environments, or domain knowledge in law, finance, medicine, or a scientific discipline.

• Submission of a specific benchmark or evaluation methodology proposal is required.

🏝️ Benefits

• Unlimited API credits.

• A dedicated budget for GPU computing resources.

• Compensation for expert/human-data time.

• Weekly one-on-one mentorship with a member of the APEX research team.

• Regular access to the wider research organization.

• Access to frontier model APIs.

• Use of Mercor’s internal evaluation infrastructure.

• Opportunities to tackle real enterprise evaluation challenges from Mercor’s clients, when applicable.

• Optional workspace in Mercor’s San Francisco office.

• Introductions to Mercor’s network of researchers across frontier labs and academia.

• Exceptional fellows may be considered for a full-time position on the APEX research team at the conclusion of the fellowship.