QA Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Develop and execute test plans for GenAI, LLM, Agentic AI, and RAG systems.

• Create and uphold automated test suites utilizing pytest and Python.

• Design and implement API tests for model endpoints, data pipelines, and microservices.

• Conduct functional, regression, and performance testing of AI services.

• Integrate testing processes into CI/CD pipelines using tools such as Jenkins, GitHub Actions, or GitLab CI.

• Evaluate model output quality, bias, and fairness metrics, and document the results.

• Collaborate with ML teams to establish test harnesses for new model releases.

• Generate and sustain test data sets, fixtures, and mock services.

• Identify defects, monitor their resolution, and verify fixes to maintain quality standards.

• Guide junior testers and advocate for best practices in AI quality assurance.

• Partner closely with data scientists, ML engineers, and product managers to validate Agentic AI and RAG workflows.

• Track performance metrics, identify anomalies in generated outputs, and maintain documentation and dashboards.

• Promote the adoption of scalable QA practices throughout the organization.

⛳️ Requirements

• Over 4 years of QA engineering experience in AI or data science settings.

• Strong expertise in pytest, Python, and API testing frameworks (e.g., Requests, RestAssured).

• Practical experience with GenAI, LLM, Agentic AI, and RAG testing methodologies.

• Familiarity with CI/CD pipelines and automation tools (Jenkins, GitHub Actions).

• In-depth knowledge of AI ethics, bias mitigation, and model explainability.

• Exceptional analytical and problem-solving abilities with a detail-oriented approach.

• Capable of translating complex AI concepts into clear, testable requirements.

• Strong communication skills for effective cross-functional collaboration.

• Proven history of delivering high-quality releases punctually.

• Enthusiasm for keeping up with emerging AI technologies and testing trends.

🏝️ Benefits

• Opportunity to work on cutting-edge AI technologies.

• Collaborative and innovative work environment.

• Professional development and growth opportunities.

• Competitive salary and comprehensive benefits package.