
AI Fact-Checking & Response Evaluation Reviewer, German
Posted Nov 11

Posted Nov 11
This is a fully remote position, open to applicants in Germany.
• Analyze and assess AI-generated responses in the German language.
• Validate factual assertions by utilizing trustworthy online resources and supportive evidence.
• Evaluate if the evidence substantiates, contradicts, or does not support the conclusions made by AI.
• Identify limitations in AI, reasoning errors, inaccuracies, and quality deficiencies.
• Assess the quality, relevance, thoroughness, and alignment of responses with user intent.
• Spot inconsistencies, unsupported assertions, contradictory data, and absent evidence.
• Utilize independent judgment instead of depending entirely on automated evaluations.
• Question AI-generated conclusions when evidence indicates an alternative result.
• Offer clear and constructive feedback along with reasoning for review decisions.
• Engage in training, calibration, and quality alignment sessions conducted in English.
• Uphold quality, consistency, commitment, and productivity standards while fulfilling project requirements.
• Work collaboratively with project teams to enhance reviewer standards and AI quality results.
• Complete guideline evaluations, platform training, UI navigation tasks, knowledge assessments, fact-checking exercises, and project simulations during the qualification phase.
• Native or near-native proficiency in German.
• Strong command of English, including the ability to comprehend guidelines and engage in training sessions.
• Prior experience in Fact-Checking, Quality Assurance, Content Review, Search Evaluation, Annotation Review, Trust & Safety, Data Quality, Linguistic QA, Research Analysis, or similar analytical roles.
• Exceptional critical-thinking and analytical capabilities.
• Strong evidence-based decision-making skills.
• High attention to detail and consistency.
• Capacity to independently assess and question AI-generated conclusions.
• Ability to differentiate between factual support, contradiction, and lack of sufficient evidence.
• Competence in interpreting complex queries and accurately evaluating response quality.
• Comfortable managing structured data and engaging with AI-related projects.
• Able to navigate ambiguity and incomplete information effectively.
• Required to pass a qualification assessment in English/German.
• Preferred: Experience in reviewing AI training data or annotation projects.
• Preferred: Background in Response Evaluation, Search Quality Rating, RLHF, LLM evaluation, AI model assessment, or human-in-the-loop review processes.
• Preferred: Experience in Fact-Checking, Trust & Safety, Information Verification, or Online Research.
• Preferred: Background in journalism, research, education, editorial review, investigation, or content quality assessment.
• Preferred: Experience in providing quality feedback, coaching, or calibration support to reviewing teams.
• Preferred: Experience with time-sensitive content such as news, public information, sports, finance, or search-related topics.
• Fully remote and flexible work environment.
• Compensation in USD.
• Involvement in cutting-edge AI and language technologies.
• Collaborate with a global network of AI quality experts.
• Long-term opportunities for high-performing contributors.
• Potential advancement into specialized roles in AI Quality, Fact-Checking, and QC Specialist positions.
• All qualification activities are unpaid.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.