Product Data Scientist – AI Evaluation, Quality

This is a fully remote position, open to applicants in Estonia.

📋 Description

• Take ownership of and enhance our offline evaluation suite across various products — including datasets (capability + regression), judges, and metrics.

• Develop and uphold online quality dashboards: tracking resolution rates, customer satisfaction (CSAT), thumbs up/down feedback, LLM-as-judge indicators, error rates, and latency.

• Close the production feedback loop by analyzing failure patterns from real traffic, transforming them into regression cases, and suggesting improvements to Product and domain experts.

• Strengthen methodology by ensuring judge stability and effectively managing non-determinism.

• Convert data into actionable insights – conduct weekly syncs, clarify trade-offs, and avoid creating dashboards that lack purpose.

⛳️ Requirements

• Proficiency in Python and SQL — capable of executing a comprehensive analysis from start to finish.

• Strong background in statistics — including sampling, hypothesis testing, variance, and comprehension of noisy metrics.

• An analytical mindset — focusing on business questions rather than just the tools.

• Over 3 years of experience in analyst or data scientist positions, with at least one role in a product-oriented context.

🏝️ Benefits

• Flexibility to travel and work remotely or in a hybrid model throughout Europe.

• Stock Options Program accessible to all team members.

• Ongoing support and commitment to ensure a successful and fulfilling experience at Finom.

• Exclusive Work & Swim Program offering corporate accommodation in Cyprus.