AI Response Labeler – French Specialty

This is a fully remote position, open to applicants in Chile.

📋 Description

• Conduct thorough side-by-side comparisons of AI-generated responses to determine the superior option.

• Assess responses for factual accuracy, relevance, completeness, clarity, reasoning, adherence to instructions, tone, and overall quality.

• Evaluate content written in English, French, or a combination of both languages.

• Review general-purpose questions and answers, web search results, file-based tasks, image-based responses, content generation requests, as well as single-turn and multi-turn conversations.

• Leverage French expertise regarding language, terminology, tone, regional conventions, idioms, and the cultural context relevant to France.

• Identify nuanced differences such as unsupported claims, incomplete reasoning, overlooked instructions, unnatural phrasing, cultural inaccuracies, and overall usefulness.

• Accurately and consistently apply detailed, scenario-specific annotation guidelines.

• Make independent evaluation decisions in ambiguous cases.

• Document evaluation decisions and provide concise, evidence-based rationale when necessary.

• Complete evaluations within expected time and productivity standards, typically a minimum of 25 tasks per day.

• Engage in training, guided practice, calibration sessions, qualification reviews, and ongoing quality review activities.

• Integrate feedback and adjust evaluations to meet team and client quality standards.

⛳️ Requirements

• Native-level or professional proficiency in French.

• Extensive knowledge of the linguistic conventions, regional vocabulary, idioms, tone, and cultural context of the French language as used in France.

• Strong fluency in English along with excellent reading comprehension.

• Solid analytical and critical thinking abilities.

• Capability to evaluate content across a variety of topics, formats, and task types.

• Proficient in assessing factual accuracy, relevance, reasoning, clarity, instruction adherence, cultural appropriateness, and overall usefulness.

• Ability to discern subtle differences in meaning, quality, tone, and user intent.

• Good judgment in applying structured evaluation criteria to ambiguous or unfamiliar situations.

• Strong written communication skills.

• Exceptional attention to detail with the ability to maintain accuracy within set timeframes.

• Ability to learn and consistently implement detailed evaluation frameworks.

• Ability to work independently while remaining in alignment with shared quality standards.

• Comfort with performing repetitive, detail-oriented tasks for extended periods while maintaining focus, accuracy, and consistent judgment.

• Capacity to accept feedback, recalibrate decisions, and adapt as evaluation criteria evolve.

• During the approximately 30-day training and qualification period, must work from 9:00 a.m. to 5:00 p.m. Pacific Time.

• Preferred: experience in side-by-side labeling, annotation, comparative content evaluation, quality assessment, evaluation of AI-generated responses, model quality assessment, data labeling or annotation, search relevance, content quality, factual accuracy, user-facing digital experiences, and detailed guidelines or rubrics.

🏝️ Benefits

• Medical, dental, and vision coverage.

• Flexible Spending Account (FSA).

• 401(k) retirement plan.

• Competitive paid time off.

• Parental leave.

• Opportunities for professional growth and development.

• Benefits in accordance with local regulations and the terms of employment through an Employer of Record partner.