Senior Quality Engineer – Chatbot

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Develop and enhance quality practices, testing strategies, and release processes for the Chatbot product.

• Create and manage an output-evaluation framework to assess Chatbot quality, reliability, and regression issues.

• Evaluate dynamic and non-deterministic LLM-based systems beyond traditional deterministic test cases.

• Identify, assess, and enhance guardrails for AI-specific risks, including prompt injection and unexpected model behavior.

• Collaborate with engineers through code reviews, technical discussions, and joint debugging efforts.

• Investigate testing deficiencies, research appropriate methods and tools, and provide evidence-based recommendations.

• Design proof-of-concepts for innovative testing methodologies or tools and assist in the production of effective solutions.

• Aid in release testing, quality signals, and criteria for release readiness.

• Contribute to performance-testing methodologies for LLM-powered workflows.

• Share expertise, facilitate technical discussions, and help build the Quality Chapter within the engineering team.

⛳️ Requirements

• Extensive hands-on experience in software quality engineering, SDET, test automation, or a closely related engineering role.

• Experience in testing chatbots, conversational AI, RAG, LLM-powered, or other dynamic and non-deterministic systems.

• Proficient in at least one programming language.

• Technical capability to comprehend production code, engage in code reviews, and collaborate effectively with software engineers.

• Experimental and investigative mindset for navigating uncertain problem spaces and translating findings into practical enhancements.

• Excellent written and verbal communication skills.

• Strong collaboration abilities and a pragmatic approach when working with Product, Engineering, and ML colleagues.

• Sound judgment in prioritizing quality work based on risk, user impact, and delivery objectives.

• Bonus: experience in AI security and safety testing.

• Bonus: experience in testing conversational AI, chatbot platforms, or agentic workflows.

• Bonus: experience as the first Quality Engineer in a team.

• Bonus: experience mentoring engineers or influencing testing practices within a cross-functional team.

• Stay updated with industry trends and encourage knowledge sharing.

🏝️ Benefits

• Competitive compensation.

• Stock options.

• Fully remote work option, with the flexibility to work from company hubs or a combination of both.

• Company-provided laptop.

• Stipend for remote setup expenses.

• Co-working budget.

• Flexible schedules and work locations.

• Generous paid time off, in addition to local public holidays.

• Enhanced parental leave.

• Health and retirement benefits.

• Annual budget for learning and development.

• Opportunities for annual workaways.

• Regular virtual and in-person social events.

• Chance to contribute to pioneering projects that shape the future of work.