Quality Evaluator – Fluent English

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Engage with various AI models and tools across a diverse array of prompts and subjects.

• Compare multiple AI-generated responses side by side.

• Evaluate accuracy, helpfulness, adherence to instructions, clarity, and overall quality of responses.

• Adhere to comprehensive rating guidelines and multi-step evaluation processes.

• Craft clear, well-reasoned justifications for evaluation decisions.

• Consistently apply evaluation criteria across different tasks.

• Contribute to enhancing the accuracy, helpfulness, and reliability of AI systems.

⛳️ Requirements

• Proficiency in English, with strong reading comprehension and writing capabilities.

• Significant hands-on experience with LLMs and the critical assessment of their outputs.

• Strong analytical judgment and the ability to evaluate multiple facets of response quality.

• Exceptional attention to detail and the capacity to follow complex instructions meticulously.

• Ability to articulate evaluation decisions clearly and persuasively.

• Comfort with switching between AI models, tools, and multi-step workflows.

• Previous experience in AI response evaluation, data annotation, or quality assurance is a plus.

• Background in editing, research, fact-checking, or analytical writing is advantageous.

• Familiarity with evaluation rubrics and side-by-side comparison tasks is desirable.

🏝️ Benefits

• Flexibility to work remotely.

• Part-time scheduling options available.

• Independent contractor status.

• Commitment period of up to 3 months.