AI Evaluator, Polish

This is a fully remote position, open to applicants in Poland.

📋 Description

• Develop and conduct brief multi-turn conversations (usually 1–5 turns) aimed at assessing AI personalization behavior.

• Construct prompts based on realistic personal scenarios to test contextual comprehension.

• Analyze AI responses to evaluate the appropriate application of personalization.

• Assess grounding quality to ensure the model does not generate unsupported claims about the user.

• Evaluate integration quality by confirming that personal signals are utilized naturally (rather than appearing forced or robotic).

• Conduct side-by-side comparisons of two responses to determine which one is more helpful, natural, and relevant.

• Compose clear and structured rationales that explain rankings and reference specific conversation turns.

• Verify debugging information to confirm the correct data sources were utilized.

• Uphold strict workflow standards, including the removal of evaluation conversations when necessary.

⛳️ Requirements

• Proficient in Polish (both reading and writing) — Polish serves as the primary language for evaluation.

• Bachelor’s degree or equivalent experience in a related field (such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or another analytical discipline).

• Strong analytical skills and the ability to evaluate nuanced AI outputs.

• Exceptional written communication abilities with a knack for producing structured evaluation notes.

• Keen attention to detail when comparing similar responses.

• Capacity to work independently in a fully remote setting.

• Access to a reliable desktop or laptop and a stable internet connection.

• Willingness to utilize your primary personal Google account and enable personal data sources for evaluation purposes.

🏝️ Benefits

• Competitive compensation package.

• Flexible working hours tailored to your schedule.

• Opportunities for professional development and growth.

• Collaborative and inclusive work environment.