Research Scientist, Benchmarks & Evals

This is a fully remote position, open to applicants in United States.

📋 Description

• A minimum of 3 years of experience in machine learning research or research engineering.

• A benchmark or evaluation that you have developed and that has been utilized by others.

• Proven expertise in training and fine-tuning image or video models using PyTorch.

• Profound understanding of experimental design and statistics, including controls, ablations, and human studies.

• Capability to conduct experiments from start to finish, from utilizing GPUs to producing the final report.

• Ability to communicate clearly in writing for both researchers and buyers.

• Publications at prestigious conferences such as NeurIPS, ICML, ICLR, or CVPR, particularly in datasets and benchmarks (this is a bonus).

• Experience working on an evaluation or data team within an AI lab (this is a bonus).

• Background in selling or licensing data to AI laboratories (this is a bonus).

• Familiarity with data valuation or data attribution research (this is a bonus).

• Proficiency in Python and PyTorch.

• Experience with open-weight image and video models.

• Knowledge of cloud GPUs.

• Experience with PostgreSQL (Supabase).

• Familiarity with S3 storage.

⛳️ Requirements

• 3+ years of ML research or research engineering experience.

• A benchmark or evaluation you built that other people actually used.

• Strong hands-on experience training and fine-tuning image or video models (PyTorch).

• Deep understanding of experimental design and statistics (controls, ablations, human studies).

• Experience running experiments end to end, from GPUs to final report.

• Clear writing for both researchers and buyers.

• Publications at NeurIPS, ICML, ICLR or CVPR (datasets and benchmarks especially) (bonus).

• Experience on an evaluation or data team at an AI lab (bonus).

• Experience selling or licensing data to AI labs (bonus).

• Experience with data valuation or data attribution research (bonus).

• Python / PyTorch.

• Open-weight image and video models.

• Cloud GPUs.

• PostgreSQL (Supabase).

• S3 storage.

🏝️ Benefits

• $350,000 – $750,000 equity.

• 0.25% – 1% equity.