Quality Assurance Engineer – Automation

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Take responsibility for the quality of the Lyrebird platform from the initial commit to the release gate.

• Enhance Playwright end-to-end test suites by minimizing runtime and eliminating flaky tests.

• Expand API and partner integration testing suites.

• Develop contract tests based on OpenAPI schemas.

• Implement coverage gates within the Buildkite pipeline.

• Assist engineering teams in CI/CD development and oversight.

• Evaluate clinical model outputs using DeepEval, openevals, and Langfuse traces.

• Design assessments for non-deterministic AI responses.

• Conduct Locust load tests prior to major releases and share findings with the SRE team.

• Create health checks, synthetic monitors, and post-deployment tests.

• Manage the release quality gate and make evidence-based approval decisions.

• Elevate testing standards through shared patterns, fixtures, and code reviews.

⛳️ Requirements

• Extensive experience in test automation using both TypeScript and Python.

• Profound hands-on expertise with Playwright.

• Experience in component testing with Vitest and Testing Library.

• Strong skills in pytest, including async tests, fixtures, mocking, and test data factories like polyfactory or Faker.

• Experience executing tests in CI pipelines, preferably with Buildkite.

• Familiarity with running tests in Docker.

• Knowledge of coverage measurement and reporting.

• Proficient understanding of SQL and Postgres.

• Confidence in testing at the database level.

• Experience in API contract testing against OpenAPI schemas.

• Authorized to work in the UK indefinitely without needing visa sponsorship now or in the future.

• Nice to have: experience in evaluating large language model outputs using DeepEval, openevals, or Langfuse.

• Nice to have: experience in load testing with Locust.

• Nice to have: familiarity with Svelte, Storybook, or Supabase and its CLI-based database testing.

• Nice to have: experience with Sentry, OpenTelemetry, and Honeycomb.

• Nice to have: background in healthcare, clinical software, or other regulated, safety-critical environments.

🏝️ Benefits

• An inclusive, safe, and supportive workplace culture.

• Opportunity to work on impactful healthcare technology that alleviates clinicians' administrative tasks.

• Chance to contribute to a diverse team.

• Encouragement for applicants from underrepresented backgrounds in tech.