Lead Software Engineer in Test – Agentic AI

This is a fully remote position, open to applicants in United States.

📋 Description

• Spearhead the quality engineering strategy and its implementation across next-generation AI-driven applications and distributed systems.

• Create, develop, and sustain scalable automation frameworks and regression testing suites for UI, API, backend, and AI-enhanced applications.

• Direct quality engineering projects for Agentic AI systems, encompassing LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms.

• Assess AI agent behaviors, including multi-agent coordination, tool invocation, prompt chaining, memory retention, conversational workflows, and fallback/retry mechanisms.

• Formulate and implement functional, integration, regression, end-to-end, and performance testing strategies.

• Collaborate with Software Engineers, Product Managers, and DevOps teams throughout the software development lifecycle.

• Develop reusable automation solutions for APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business operations.

• Design and manage test environments, CI/CD integrations, cloud execution platforms, and automation pipelines.

• Establish best practices for code quality, maintainability, scalability, dependency injection, observability, and testability.

• Verify AI workflow resilience, distributed system reliability, data consistency, and orchestration performance under failure and edge-case scenarios.

• Mentor QA engineers and contribute to QA standards, automation strategies, reporting practices, and continuous improvement efforts.

• Conduct activities in accordance with EEO laws, HIPAA, ERISA, and other relevant regulations.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Management Information Systems, Engineering, or a related technical discipline.

• 8+ years of experience in Software Development or Software Engineer in Test positions.

• 5+ years of hands-on experience in developing automation frameworks using JavaScript/TypeScript or Java.

• 2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.

• Proven experience in validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes.

• Familiarity with AI orchestration frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies.

• Strong proficiency in API testing, backend integration testing, UI automation, and end-to-end workflow validation.

• Knowledge of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles.

• Experience in cloud-native and containerized environments including Azure, AWS, GCP, Docker, and Kubernetes.

• Background in integrating automated test suites into CI/CD pipelines and distributed cloud environments.

• Experience validating asynchronous, event-driven, and distributed systems with an emphasis on resilience and fault tolerance.

• Strong analytical, debugging, troubleshooting, and problem-solving capabilities.

• Excellent organizational, communication, and stakeholder collaboration skills in Agile settings.

• Preferred: experience with AI observability, telemetry, RAG pipeline validation, vector databases, or prompt evaluation frameworks.

• Preferred qualifications include familiarity with Azure DevOps or GitHub Actions, healthcare systems and HIPAA compliance, event-driven architectures, messaging systems, distributed data platforms including SQL Server, QA team leadership, mentoring, and enterprise test reporting.

🏝️ Benefits

• Generous annual bonus potential.

• 401(k) with Employer Match.

• Flexible Time Off.

• 11 paid holidays.

• "Your Day Your Way" cultural or social significance day.

• Options for Medical, Dental, and Vision plans.

• Health Savings Account with employer match.

• Flexible Spending Account.

• 100% Company-Paid Parental leave after 6 months of employment.

• 100% Company-Paid Life Insurance.

• 100% Company-Paid Short/Long Term Disability Insurance.

• Nextech Luminary Peer Recognition Program.

• Wellness Program including discounts on medical premiums.

• Employee Assistance Program offering free counseling sessions.

• Corporate Discounts on Retail, Travel, and Entertainment.