Study Domain Expert – Judge Calibration

This is a fully remote position, open to applicants in United States.

📋 Description

• Utilize professional judgment to assess realistic Sales tasks and evaluate outputs generated by models.

• Sign a non-disclosure agreement prior to receiving study materials.

• Execute one realistic Sales assignment using provided source documents.

• Analyze model results, assign rankings and scores, and compose evidence-based justifications.

• Offer structured feedback regarding the task and overall study design.

• Fulfill 13 hours of work between September 12–15, 2026.

• Engage in projects aimed at training and improving AI systems.

⛳️ Requirements

• Over 5 years of experience in Sales or a closely related field.

• Experience at a prestigious firm is preferred.

• Ideally, have held a managerial, director, or principal-level Sales position; entry-level experience is not sufficient.

• Must be fluent or a native English speaker.

• Previous work as an expert is advantageous.

• Required to sign a non-disclosure agreement before receiving study materials.

• No use of large language models or other AI assistants during the study.

• Individuals who have participated in Project Alchemy in any capacity or accessed its source-world data are ineligible.

• Must operate as an independent contractor.

• H1-B and STEM OPT candidates are not supported.

• No geographical limitations apply.

🏝️ Benefits

• Fully remote position.

• Flexible working hours; tasks can be completed according to your own schedule.

• Weekly payments through Stripe or Wise based on services provided.

• Competitive compensation.

• Opportunity to collaborate with leading researchers.

• Chance to contribute to the development of the next generation of AI systems.

• Referral bonuses of up to $360 for each successful referral.

• Reasonable accommodations available upon request.