Software Engineer – AI Evaluation

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Develop and create the AI evaluation suite designed to assess and enhance the quality of AI outputs.

• Work across the entire stack, including data pipelines, backend services, and front-end tools.

• Collaborate with prompt engineers and subject matter experts to transform knowledge into tests, metrics, and tools.

• Assist in defining practices for measuring AI quality.

• Automate evaluation processes and results within the software development lifecycle.

• Make contributions over time to AI-driven products.

• Launch an evaluation suite utilized by engineering teams within three months.

• Expand evaluations across all AI products and integrate them into the SDLC within six months.

• Establish evaluation as a standard aspect of AI delivery and become a trusted authority on AI quality within twelve months.

⛳️ Requirements

• Upper mid-level to senior-level professional experience in software engineering.

• Proven experience in creating AI-powered products or tools.

• Full-stack experience that encompasses both front-end and back-end.

• Ability to work independently and navigate ambiguity, while setting your own direction.

• Experience in collaborating with non-engineers and converting their expertise into functional software.

• Strong emphasis on quality, testing, and meticulous attention to detail.

• Familiarity with AI coding tools in everyday work.

• Proficient in both written and spoken English.

• Nice to have: experience in developing evaluation suites for AI products.

• Nice to have: proficiency in Python and TypeScript.

• Nice to have: background in building internal tools, testing frameworks, or data pipelines.

• Nice to have: experience working at a startup or in a small team, managing projects from inception to completion.

🏝️ Benefits

• Pension scheme.

• Personal performance bonus.

• 26 days of holiday each calendar year.

• Bupa health & dental coverage.

• Group life assurance.

• Employee Assistance Program (EAP).

• Cycle to work scheme.

• Commitment to diversity, equity, and inclusion.

• Reasonable adjustments accommodated during the hiring process.