AI Response Labeler – Castilian Spanish Specialty

This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct side-by-side evaluations of AI-generated responses to determine which response is more effective.

• Assess responses for factual correctness, relevance, completeness, clarity, reasoning, adherence to instructions, tone, and overall quality.

• Review content written in English, Castilian Spanish, or a combination of both languages.

• Evaluate a variety of content types including general-purpose questions and answers, web search results, file-based tasks, image-based responses, content generation requests, as well as single-turn and multi-turn conversations.

• Utilize expertise in Castilian Spanish to evaluate language, terminology, tone, regional conventions, idioms, and cultural context specific to Spain.

• Focus on the overall quality of responses rather than merely assessing grammar, translation, or fluency.

• Identify unsupported assertions, incomplete reasoning, overlooked instructions, awkward phrasing, cultural inaccuracies, and variances in usefulness.

• Accurately and consistently apply scenario-specific annotation guidelines.

• Make independent evaluation decisions in cases where ambiguity exists.

• Document evaluation decisions and provide clear, evidence-based reasoning.

• Complete evaluations within set timeframes and productivity standards.

• Maintain consistent judgment across a high volume of diverse assignments.

• Engage in training, guided practice, calibration sessions, qualification reviews, and quality assessment activities.

• Integrate feedback and adjust evaluation decisions to align with team and client quality standards.

• Achieve a minimum of 25 tasks daily, with the majority of tasks taking around 15 minutes each.

• Successfully complete an approximately 30-day onboarding and qualification program prior to beginning production work.

⛳️ Requirements

• Native-level or professional proficiency in Castilian Spanish.

• Extensive knowledge of linguistic conventions, regional vocabulary, idioms, tone, and cultural context of Spanish as utilized in Spain.

• Strong fluency in English with excellent reading comprehension, including the ability to understand complex prompts, AI-generated responses, and detailed annotation guidelines written in English.

• Robust analytical and critical thinking skills extending beyond language evaluation.

• Capability to evaluate content across a wide range of topics, formats, and task types.

• Proficiency in assessing factual accuracy, relevance, reasoning, clarity, adherence to instructions, cultural appropriateness, and overall usefulness.

• Ability to discern subtle differences in meaning, quality, tone, and user intent.

• Sound judgment in applying structured evaluation criteria to ambiguous or unfamiliar situations.

• Excellent written communication skills with the ability to clearly and concisely explain evaluation decisions.

• Exceptional attention to detail and the capacity to maintain accuracy within specified time expectations.

• Ability to learn and consistently apply intricate evaluation frameworks.

• Capability to work independently while staying aligned with shared quality standards.

• Comfort in performing repetitive, detail-oriented tasks over extended periods while ensuring focus, accuracy, and consistent judgment.

• Ability to accept feedback, recalibrate decisions, and adapt as evaluation guidelines change.

• Preferred experience in side-by-side labeling, annotation, comparative content evaluation, or quality assessment.

• Preferred experience in evaluating AI-generated responses or contributing to model quality assessment.

• Preferred experience with data labeling or annotation.

• Preferred experience evaluating search relevance, content quality, factual accuracy, or user-facing digital experiences.

• Preferred experience working with detailed guidelines, rubrics, or structured decision-making frameworks.

• Candidates must be authorized to work in the United States and reside within a U.S. time zone.

🏝️ Benefits

• Medical, dental, and vision insurance coverage.

• Flexible Spending Account (FSA).

• 401(k) retirement plan.

• Competitive paid time off.

• Parental leave.

• Opportunities for professional growth and development.