QA Lead – Agentic AI Platform

This is a fully remote position, open to applicants in Canada.

📋 Description

• Oversee comprehensive quality assurance for a cutting-edge Agentic AI platform.

• Establish quality benchmarks and design automated evaluation systems.

• Direct testing strategies and ensure preparedness for the release of AI-driven conversational solutions.

• Develop evaluation harnesses and formulate test plans.

• Perform regression testing and assess data quality.

• Conduct AI safety assessments and prompt injection tests.

• Manage defect triage and supervise a team of QA and Data Testers.

• Collaborate closely with engineering, product, architecture, and data teams to guarantee the delivery of dependable and high-quality AI solutions.

⛳️ Requirements

• Proven leadership in QA.

• Proficiency in Python.

• Experience with test automation.

• Expertise in AI/LLM testing.

• Knowledge of conversational AI testing.

• Familiarity with Retrieval-Augmented Generation (RAG) testing.

• Understanding of LLM evaluation frameworks.

• Experience with golden dataset and regression testing.

• Skills in intent and context validation.

• Proficiency in multi-turn dialogue testing.

• Expertise in data quality testing.

• Knowledge of prompt injection and AI safety testing.

• Experience with release readiness and quality gates.

• Proficient in defect management and test strategy.

• Familiarity with CI/CD integrated testing.

• Strong team leadership and mentoring abilities.

🏝️ Benefits

• Flexible working hours.