Quality Evaluator – Fluent English

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Engage with various AI models and tools across a diverse array of prompts and subjects.

• Conduct side-by-side comparisons of multiple AI-generated responses.

• Evaluate the accuracy, usefulness, adherence to instructions, clarity, and overall quality of responses.

• Adhere to comprehensive rating guidelines and follow multi-step evaluation procedures.

• Compose clear and well-reasoned justifications to support evaluation decisions.

• Consistently apply evaluation criteria across different tasks.

• Contribute to enhancing the accuracy, usefulness, and reliability of AI systems.

⛳️ Requirements

• Proficiency in English, with exceptional reading comprehension and writing abilities.

• Significant hands-on experience utilizing LLMs and critically analyzing their outputs.

• Strong analytical skills and the ability to assess multiple aspects of response quality.

• Keen attention to detail and the ability to meticulously follow intricate instructions.

• Capability to articulate evaluation decisions clearly and persuasively.

• Comfort in transitioning between AI models, tools, and multi-step workflows.

• Previous experience in AI response evaluation, data annotation, or quality assurance (preferred).

• Background in editing, research, fact-checking, or analytical writing (preferred).

• Familiarity with evaluation rubrics and tasks involving side-by-side comparisons (preferred).

🏝️ Benefits

• Opportunity for remote work.

• Part-time independent contractor arrangement.

• Commitment duration of up to 3 months.