AI Response Labeler – French Specialty

This is a fully remote position, open to applicants in Costa Rica.

📋 Description

• Conduct side-by-side assessments of AI-generated replies to determine which response is more effective.

• Analyze responses for factual correctness, relevance, completeness, clarity, reasoning, adherence to instructions, tone, and overall quality.

• Review content written in English, French, or a combination thereof.

• Evaluate general-purpose inquiries and answers, web-search results, tasks based on files, image-based replies, content-generation requests, and both single-turn and multi-turn dialogues.

• Utilize French expertise concerning language, terminology, tone, regional customs, idioms, and cultural context specific to France.

• Identify unsupported assertions, incomplete reasoning, overlooked instructions, awkward phrasing, cultural discrepancies, and variations in usefulness.

• Accurately and consistently apply detailed scenario-specific annotation guidelines.

• Make independent evaluation judgments and provide concise, evidence-supported reasoning.

• Complete evaluations within set time and productivity standards; typically accomplish a minimum of 25 tasks daily.

• Engage in training, guided practice, calibration sessions, qualification assessments, and ongoing quality review activities.

• Integrate feedback and modify evaluation decisions to meet team and client quality benchmarks.

• Maintain consistent judgment across a high volume of diverse assignments.

⛳️ Requirements

• Native-level or professional proficiency in French.

• Extensive knowledge of French linguistic norms, regional lexicon, idioms, tone, and cultural context pertinent to France.

• Strong fluency in English and excellent reading comprehension skills.

• Robust analytical and critical-thinking capabilities.

• Capacity to evaluate content across various subjects, formats, and task types.

• Ability to assess factual accuracy, relevance, reasoning, clarity, adherence to instructions, cultural appropriateness, and overall effectiveness.

• Skill in recognizing subtle distinctions in meaning, quality, tone, and user intent.

• Sound judgment when applying structured evaluation criteria to ambiguous or unfamiliar situations.

• Strong written communication abilities.

• Exceptional attention to detail and the ability to maintain accuracy within established timeframes.

• Capability to learn and consistently apply intricate evaluation frameworks.

• Ability to work autonomously while remaining aligned with collective quality standards.

• Comfort with performing repetitive, detail-oriented tasks over extended periods.

• Ability to accept feedback, adjust decisions, and adapt as evaluation criteria evolve.

• All new hires must successfully finish structured onboarding and qualification prior to production work.

• Preferred: experience in side-by-side labeling, annotation, comparative content evaluation, quality assessment, evaluating AI-generated responses, data labeling, search relevance, content quality, factual correctness, user-facing digital experiences, or detailed guidelines/rubrics.

🏝️ Benefits

• Medical, dental, and vision insurance.

• Flexible Spending Account (FSA).

• 401(k) retirement plan.

• Competitive paid time off.

• Parental leave.

• Opportunities for professional growth and development.

• Benefits in accordance with local regulations and the terms of employment.