QA Engineer, AI – Platform Infrastructure

This is a fully remote position, open to applicants in California.

📋 Description

• Serve as a Subject Matter Expert in the design, maintenance, and execution of both automated and manual testing frameworks for backend APIs, frontend components, containerized services, and workflows powered by AI/LLM.

• Design, implement, and sustain QA pipelines integrated with CI/CD processes utilizing Docker Compose and Kubernetes environments.

• Execute performance and load testing for containerized and GPU-enabled applications, pinpointing bottlenecks and potential reliability issues.

• Ensure the accuracy of AI workflows, including LangChain pipelines, CrewAI task orchestration, and LlamaIndex document processing flows.

• Create internal testing tools and frameworks using TypeScript, Python, or Bun to enhance platform reliability and accelerate developer productivity.

• Reproduce, isolate, and document defects with comprehensive context, collaborating closely with developers to facilitate swift resolutions.

• Consult and work alongside Subject Matter Experts from Engineering, Product, DevOps, and AI/ML teams to establish quality standards and formulate test plans for new services and UI components.

• Direct the development, modification, and upkeep of QA documentation, which includes test plans, test case libraries, defect reports, and quality metrics dashboards.

• Influence cross-functional teams regarding testability, technical feasibility, and quality requirements throughout the design and development phases.

• Support the planning and execution of QA-driven process enhancements and tooling upgrades in partnership with cross-functional teams.

• Oversee integration, regression, and acceptance testing for new features and changes in infrastructure.

• Offer technical mentoring and guidance to less experienced engineers on testing best practices, tooling, and defect management.

• Develop, revise, and comply with Standard Operating Procedures (SOPs), process improvements, and the standardization of templates.

• Undertake ad-hoc and cross-functional tasks and/or projects assigned to address business needs and foster developmental opportunities.

⛳️ Requirements

• A Bachelor’s degree in computer science, Software Engineering, or a closely related field is mandatory.

• A minimum of 3 years of experience in software QA or test automation is essential.

• Proficiency in containerized environments such as Docker, Docker Compose, and Kubernetes is required.

• Experience with Python and TypeScript for developing tests and automation scripts is necessary.

• Familiarity with modern web frameworks (React, Next.js, Svelte) and E2E/UI testing tools like Playwright or Cypress is required.

• Understanding of CI/CD workflows (e.g., GitHub Actions, Jenkins, or similar) is a must.

• Experience testing AI systems, particularly with LangChain, CrewAI, or LlamaIndex, is preferred.

• Background in performance testing or benchmarking for AI-driven applications is preferred.

🏝️ Benefits

• Options for Medical, Dental, and Vision Plans.

• Health and Financial Wellness Programs.

• Employer Assistance Program (EAP).

• Company-paid and voluntary Life/AD&D, Short-Term and Long-Term Disability coverage.

• Flexible Spending Accounts for Healthcare and Dependent Care.

• 401(k) Retirement Plan featuring Company Match.

• 529 Education Savings Program.

• Voluntary Legal Services, Identity Theft Protection, Pet Insurance, and Employee Discounts, Rewards, and Perks.

• Paid Time Off (PTO) that includes 11 Holidays.

• Exempt Employees qualify for Unlimited PTO.

• Non-Exempt Employees are entitled to 10 Vacation Days, 56 Hours of Health Pay, 2 Personal Days, and 1 Cultural Day.