Senior Test Automation Engineer, AI-Native

This is a fully remote position, open to applicants in Germany.

📋 Description

• Develop and manage a contemporary, AI-driven testing strategy for our upcoming product iteration (Laravel/Vue.js)

• Implement comprehensive test automation using Playwright or similar tools, which includes AI-assisted generation of test cases and self-healing tests

• Ensure quality assurance for our AI products, focusing on the evaluation of LLM outputs, prompt testing, hallucination verification, and behavioral assessments for agentic workflows

• Create a testing pyramid in collaboration with developers (Unit, Integration, E2E) ensuring seamless integration into CI/CD

• Facilitate AI-assisted bug reproduction and root-cause analysis; convert customer support tickets into replicable tests

• Engage closely with developers, product managers, and the Co-CEO

⛳️ Requirements

• Familiarity with an AI-native toolset (tools you actively utilize): test automation with Playwright, Cypress, or similar frameworks, preferably with AI code generation capabilities

• Regular use of tools like Claude or ChatGPT for generating test cases, analyzing edge cases, and creating test code

• Proficiency with Cursor or similar AI IDEs to swiftly comprehend Vue.js and Laravel code and develop suitable tests

• Ideally possess experience with LLM evaluation (Promptfoo, Braintrust, LangSmith, or custom evaluation processes)

• Knowledge of MCP, agentic workflows, or Custom GPTs and the systematic approach to testing these elements

• You recognize the distinction: knowing when an AI-generated test is adequate and when hands-on testing with precise assertions is necessary

• Experience with CI/CD (GitHub Actions, GitLab CI), Git, and workflows driven by pull requests

• Strong grasp of API testing (REST), databases (MySQL/Postgres), and web architectures. Experience with mobile applications (iOS & Android) is a plus.

🏝️ Benefits

• Complete access to all necessary AI tools (Claude Max, Cursor, Playwright, or any subscription of your choice)

• 30 days of vacation along with permanent employment contracts

• A remote-first company with flexible working hours — work from anywhere in Germany!

• Regular workshops, workations, and team-building activities

• Assistance for home office setup, selection of laptop, and a monthly voucher for various benefits or gym memberships, plus a company bike

• The advantages, freedom, and influence of a startup paired with the stability of a medium-sized company that has been established for over 25 years