SDET 3

This is a fully remote position, open to applicants in India.

📋 Description

• Take ownership of the test strategy and establish quality criteria for the product area.

• Develop and sustain UI automation, API automation, and comprehensive end-to-end regression suites.

• Create evaluation strategies for AI agents utilizing LLM-as-judge, rubrics, and various evaluation methods.

• Construct evaluation datasets and automated assessment suites in Python, integrated with CI/CD processes.

• Assess AI agents for precision, consistency, tool utilization, safety, and data management.

• Troubleshoot agent malfunctions using LangSmith and comparable AI observability tools.

• Evaluate quality from release to release and detect regressions.

• Analyze product requirements and pinpoint deficiencies in requirements and test coverage.

• Examine test code, establish testing standards, and mentor SDETs.

• Convey findings and suggestions to engineering, product teams, and leadership.

• Collaborate closely with GenAI engineering, CX engineering, and product teams.

⛳️ Requirements

• 6–9 years of experience in software testing, including hands-on experience as an SDET.

• Proficient knowledge of Python and Java.

• Proven experience in owning test strategies and enhancing automation and release quality.

• Familiarity with UI automation tools such as Selenium or Playwright.

• Experience in developing API automation and end-to-end frameworks with CI/CD integration.

• Ability to analyze product requirements and detect testing gaps.

• Previous experience in testing chatbots, IVRs, and GenAI agents.

• Expertise in designing evaluations, rubrics, and LLM-as-judge assessments.

• Working knowledge of LangChain, LangGraph, and LangSmith or similar tools.

• Experience in creating evaluation harnesses or tools for AI agents.

• Strong analytical abilities and communication skills.

🏝️ Benefits

• Flexible remote work arrangement.

• Recognized as a Certified Great Place to Work engineering center.