Quality Evaluator, Fluent English

This is a fully remote position, open to applicants in Estonia.

📋 Description

• Engage with various AI models and tools across a diverse array of prompts and subjects.

• Conduct side-by-side comparisons of multiple AI-generated responses.

• Evaluate the accuracy, helpfulness, adherence to instructions, clarity, and overall quality of responses.

• Adhere to comprehensive rating guidelines and multi-step evaluation processes.

• Provide clear and well-reasoned justifications for evaluation decisions.

• Consistently apply evaluation criteria throughout all tasks.

⛳️ Requirements

• Proficiency in English, with strong reading comprehension and writing abilities.

• Significant hands-on experience with LLMs and a critical approach to assessing their outputs.

• Exceptional analytical judgment with the capacity to evaluate multiple aspects of response quality.

• High attention to detail and the ability to meticulously follow complex instructions.

• Ability to articulate evaluation decisions in a clear and persuasive manner.

• Comfort in transitioning between different AI models, tools, and multi-step workflows.

• Previous experience in AI response evaluation, data annotation, or quality assurance is a plus.

• Background in editing, research, fact-checking, or analytical writing is advantageous.

• Familiarity with evaluation rubrics and tasks involving side-by-side comparisons is a bonus.

🏝️ Benefits

• Opportunity for remote work.

• Arrangement as a part-time independent contractor.

• Commitment duration of up to 3 months.