Lead Software Engineer in Test – Agentic AI

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, create, and uphold scalable automation frameworks and regression testing suites across UI, API, backend, and AI-driven applications.

• Spearhead quality engineering projects for Agentic AI systems, encompassing LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms.

• Assess intricate AI agent behaviors including multi-agent coordination, tool utilization, prompt chaining, memory retention, conversational workflows, and fallback/retry mechanisms.

• Formulate and implement strategies for functional, integration, regression, end-to-end, and performance testing across cloud-native and distributed settings.

• Partner with Software Engineers, Product Managers, and DevOps teams to weave testing into the software development lifecycle.

• Construct reusable automation solutions to verify APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business processes.

• Design and maintain stable test environments, CI/CD integrations, cloud execution platforms, and automation pipelines to enhance efficiency and reliability.

• Set and promote best practices concerning code quality, maintainability, scalability, dependency injection, observability, and testability.

• Direct validation efforts for AI workflow resilience, distributed system reliability, data consistency, and orchestration performance under failure and edge-case scenarios.

• Mentor QA engineers and aid in the development of scalable QA standards, automation strategies, reporting practices, and continuous improvement initiatives.

⛳️ Requirements

• Bachelor’s degree in Computer Science, MIS, Engineering, or a related technical discipline.

• Over 8 years of experience in Software Development or as a Software Engineer in Test (SDET/SET) roles.

• More than 5 years of practical experience in developing automation frameworks using JavaScript/TypeScript or Java.

• Strong proficiency with automation tools such as Playwright, Cypress, Postman, K6, or similar frameworks.

• At least 2 years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.

• Practical experience in validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes.

• Familiarity with testing AI orchestration frameworks like LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies.

• Extensive expertise in API testing, backend integration testing, UI automation, and end-to-end workflow validation.

• Comprehensive understanding of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles.

• Experience in cloud-native and containerized environments (Azure, AWS, GCP, Docker, Kubernetes).

• Proven track record of integrating automated test suites into CI/CD pipelines and distributed cloud environments.

• Experience in validating asynchronous, event-driven, and distributed systems with a focus on resilience and fault tolerance.

• Strong analytical, debugging, troubleshooting, and problem-solving capabilities.

• Excellent organizational, communication, and stakeholder collaboration skills within Agile environments.

• Experience with AI observability, telemetry, RAG pipeline validation, vector databases, or prompt evaluation frameworks (preferred).

🏝️ Benefits

• Generous annual bonus opportunity

• 401(k) with Employer Match

• Flexible Time Off: take time off when you need it without worrying about available hours

• 11 paid holidays

• Your Day Your Way - Celebrate a day of cultural or social significance to you

• Insurance: Choice of Medical, Dental, and Vision plans

• Health Savings Account with employer match

• Flexible Spending Account

• 100% Company-Paid Parental leave (After 6 months with the company)

• 100% Company-Paid Life Insurance and Short/Long Term Disability Insurance

• Nextech Luminary Peer Recognition Program

• Wellness Program including discounts on medical premiums

• Employee Assistance Program with free counseling sessions available

• Corporate Discounts on Retail, Travel, and Entertainment