AI Quality and Automation Specialist

This is a fully remote position, open to applicants in Poland, +4 more countries.

📋 Description

• Develop and sustain API, integration, and comprehensive regression tests.

• Assess AI agents based on task execution, quality of responses, tool utilization, and permissions.

• Analyze the impact of changes in models, prompts, and context.

• Implement quality assurance checks within CI/CD pipelines.

• Transform production defects into regression test scenarios.

• Leverage AI agents to navigate workflows, create tests, and replicate bugs.

• Confirm outcomes from AI-generated tests and bug reproductions.

• Disseminate effective tools and strategies with the broader AI team and central QA unit.

⛳️ Requirements

• Proficient in test automation and programming with TypeScript or Python.

• Extensive hands-on experience with web applications, APIs, Git, and CI/CD processes.

• Methodical approach to evaluating AI outputs against defined, quantifiable standards.

• Capability to quickly reproduce, isolate, and identify the root causes of bugs.

• Strong communication and documentation abilities.

• Bonus: experience with agentic browser testing, LLM evaluation frameworks, Supabase, Vercel, or GCP.

• Must currently reside in one of the eligible locations: Poland, Georgia, Bulgaria, Isle of Man, or Romania.

🏝️ Benefits

• Chance to work at the forefront of AI agent evaluation.

• Genuine ownership and accountability.

• Open culture facilitated by Slack, Monday.com, and Google Meet.

• Cameras-on during Google Meet calls.

• A diverse and inclusive workplace.

• Autonomy to innovate and take charge.