AI Fact-Checking, Response Evaluation Reviewer – Portuguese

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Utilize independent judgment to pinpoint AI blind spots, reasoning errors, factual inaccuracies, contradictory evidence, and quality deficiencies.

• Review and assess AI-generated responses in Portuguese.

• Verify factual assertions using trusted online sources and supporting evidence.

• Evaluate whether the evidence corroborates, contradicts, or fails to support AI-generated conclusions.

• Identify gaps in reasoning, unsupported claims, contradictory information, missing evidence, and quality issues.

• Assess the quality, relevance, completeness, and alignment of responses with user intent.

• Contest AI-generated conclusions when evidence indicates an alternative outcome.

• Provide clear quality feedback and justification for review decisions.

• Participate in training, calibration, and quality alignment sessions conducted in English.

• Uphold quality, consistency, commitment, and productivity expectations while meeting project requirements.

• Collaborate with project teams to enhance reviewer standards and improve AI quality outcomes.

⛳️ Requirements

• Native or near-native fluency in Portuguese.

• Strong proficiency in English, including the ability to comprehend guidelines and engage in training sessions.

• Prior experience in Fact-Checking, Quality Assurance, Content Review, Search Evaluation, Annotation Review, Trust & Safety, Data Quality, Linguistic QA, Research Analysis, or similar analytical positions.

• Exceptional critical-thinking and analytical abilities.

• Strong evidence-based decision-making skills.

• High attention to detail and consistency.

• Capability to independently assess and challenge AI-generated conclusions.

• Ability to differentiate between factual support, contradiction, and insufficient evidence.

• Proficient in interpreting complex queries and accurately evaluating response quality.

• Comfortable working with structured data and AI-related projects.

• Ability to navigate ambiguity and incomplete information.

• Must pass a qualification assessment in English/Portuguese.

• Preferred experience reviewing AI training data or annotation projects.

• Experience in Response Evaluation, Search Quality Rating, RLHF, LLM evaluation, AI model assessment, or human-in-the-loop review workflows is preferred.

• Preferred background in Fact-Checking, Trust & Safety, Information Verification, or Online Research.

• Preferred experience in journalism, research, education, editorial review, investigation, or content quality assessment.

• Previous experience providing quality feedback, coaching, or calibration support to reviewer teams is preferred.

• Experience working with time-sensitive content such as news, public information, sports, finance, or search-related content is preferred.

• Qualification activities may encompass guideline reviews, platform training, UI navigation exercises, knowledge assessments, fact-checking tasks, and project simulation exercises.

• All qualification activities are unpaid and constitute part of the candidate evaluation process.

🏝️ Benefits

• 100% remote and flexible work environment.

• Compensation in USD.

• Opportunity to work on cutting-edge AI and language technologies.

• Collaborate with a global network of AI quality professionals.

• Long-term opportunities available for high-performing contributors.

• Potential for growth into advanced roles in AI Quality, Fact-Checking, and QC Specialist positions.

• Contribute to shaping how AI systems assess information and make decisions.

• Directly influence the development of safer, more reliable, and more trustworthy AI systems.