Product Data Scientist – AI Evaluation, Quality

This is a fully remote position, open to applicants in Greece.

📋 Description

• Take ownership of and enhance our offline evaluation suite across various products — including datasets (capability + regression), judges, and metrics.

• Develop and sustain online quality dashboards that track resolution rates, customer satisfaction (CSAT), thumbs up/down feedback, LLM-as-judge signals, error rates, and latency.

• Close the feedback loop in production: analyze failure patterns from actual traffic, convert them into regression cases, and recommend solutions to product and domain experts.

• Strengthen methodology by ensuring judge stability and effectively handling non-determinism.

• Convert data into actionable decisions — conduct weekly meetings, outline clear trade-offs, and avoid creating dashboards without purpose.

⛳️ Requirements

• Proficiency in Python and SQL — capable of conducting an analysis from start to finish.

• Strong background in statistics — including knowledge of sampling, hypothesis testing, and variance.

• Analytical mindset — begin with the business question rather than the tools at your disposal.

• Over 3 years of experience in analyst or data scientist positions, with at least one role in a product-focused environment.

🏝️ Benefits

• Make a meaningful contribution to the product.

• Join us on our upward path and advance your career alongside us.

• Work in the EU with the flexibility to travel and options for remote or hybrid work throughout Europe.

• Become a stock options holder through our Stock Options Program.

• Experience our unwavering support and care, reflecting our dedication to your well-being and success.

• Participate in the Work & Swim program, which includes a month in a corporate apartment in Cyprus.

• Equal Opportunity Statement that embraces diversity.