
AI Session Annotator, English
Posted Aug 27

Posted Aug 27
This is a fully remote position, open to applicants in United States.
• Analyze recorded interactions between users and an AI assistant while capturing video through their camera.
• Assess assistant responses in relation to the visible video content.
• Detect any hallucinated or fabricated visual elements.
• Rate the sessions based on visual interpretation, accuracy, relevance, response timing, and the use of spatial/directional language.
• Determine if safety warnings regarding health, physical safety, or financial security were issued in a timely and appropriate manner.
• Provide detailed comments for each score, supported by concrete evidence.
• Document conversations turn by turn in the intake form.
• Complete a brief calibration task upon request.
• Participate in projects focused on the training and enhancement of AI models.
• Native or near-native proficiency in English, both spoken and written.
• Ability to write clear and precise English.
• Ownership of an Android phone equipped with a functioning camera and microphone (mandatory).
• Strong attention to detail.
• Capability to consistently apply a rubric.
• Preferred: Previous experience in data annotation, linguistic quality assurance, content moderation, or AI model evaluation.
• Preferred: Educational background in linguistics, translation, or interpreting.
• Skill in identifying unnatural or non-native English and regional dialects.
• Status as an independent contractor.
• Must not require H1-B or STEM OPT sponsorship.
• Weekly payments processed via Stripe or Wise based on services rendered.
• Fully remote work environment.
• Flexible work schedule, allowing you to set your own hours.
• Competitive compensation.
• Opportunity to collaborate with leading researchers in the field.
• Chance to contribute to the development of the next generation of AI systems.
• Referral bonuses of up to $120 for each successful referral.
The College Board
Mercor
Mercor
Get handpicked remote jobs straight to your inbox weekly.