Quality Evaluator, Fluent English

This is a fully remote position, open to applicants in Poland.

📋 Description

• Engage with various AI models and tools across a diverse array of prompts and subjects.

• Conduct side-by-side comparisons of multiple AI-generated outputs.

• Evaluate the accuracy, usefulness, adherence to instructions, clarity, and overall quality of responses.

• Adhere to detailed rating criteria and follow multi-step evaluation processes.

• Provide clear, well-reasoned justifications for evaluation decisions.

• Consistently apply evaluation standards across different tasks.

• Contribute to enhancing the accuracy, helpfulness, and reliability of AI systems through assessment and feedback.

⛳️ Requirements

• Proficiency in English, with strong reading comprehension and writing capabilities.

• Significant practical experience utilizing LLMs and critically evaluating their outputs.

• Strong analytical skills and the ability to assess various aspects of response quality.

• Excellent attention to detail and the capability to meticulously follow intricate instructions.

• Ability to articulate evaluation decisions clearly and persuasively.

• Comfort in transitioning between different AI models, tools, and multi-step workflows.

• Previous experience in AI response evaluation, data annotation, or quality assurance is a plus.

• Experience in editing, research, fact-checking, or analytical writing is advantageous.

• Familiarity with evaluation rubrics and tasks involving side-by-side comparisons is beneficial.

• Required application materials consist of a resume, GitHub profile, LinkedIn profile, and phone number.

🏝️ Benefits

• Opportunity for remote work.

• Part-time independent contractor arrangement.

• Commitment of up to 3 months.