Quality Evaluator, Fluent English

This is a fully remote position, open to applicants in Colombia.

📋 Description

• Engage with various AI models and tools across a diverse array of prompts and subjects.

• Analyze and compare multiple AI-generated responses concurrently.

• Evaluate accuracy, usefulness, adherence to instructions, clarity, and overall quality of responses.

• Adhere to comprehensive rating guidelines and multi-step evaluation processes.

• Compose clear and well-reasoned justifications for evaluation choices.

• Consistently apply evaluation criteria across different tasks.

• Contribute to enhancing the accuracy, usefulness, and dependability of AI systems.

⛳️ Requirements

• Proficiency in English, with excellent reading comprehension and writing capabilities.

• Significant hands-on experience with LLMs and a critical approach to assessing their outputs.

• Strong analytical skills and the ability to evaluate multiple aspects of response quality.

• Exceptional attention to detail and the capability to follow intricate instructions meticulously.

• Ability to articulate evaluation decisions clearly and persuasively.

• Comfort in navigating between AI models, tools, and multi-step workflows.

• Prior experience in AI response evaluation, data annotation, or quality assurance (preferred).

• Background in editing, research, fact-checking, or analytical writing (preferred).

• Familiarity with evaluation rubrics and side-by-side comparison tasks (preferred).

🏝️ Benefits

• Flexible remote working options.

• Part-time scheduling available.

• Independent contractor arrangement.

• Opportunity for up to 3 months of work.