
QA Automation Engineer / SDET – AI Prototyping
Posted 10 hours ago

Posted 10 hours ago
This is a fully remote position, open to applicants in Costa Rica.
• Develop and implement a practical, risk-focused testing strategy across multiple concurrently built prototypes.
• Create reusable automated checks for APIs, user interfaces, and generated documents, integrating them into continuous integration (CI) systems.
• Collaborate with the on-site Product Owner and client engineers to maintain versioned reference (golden) datasets.
• Design validation checks for AI outputs, including schema validation, defined acceptance criteria, and regression tracking.
• Emphasize deterministic checks for calculations and code compliance, ensuring they remain independent from the code being tested.
• Utilize AI-assisted test generation while thoroughly reviewing and overseeing each test produced.
• Expand regression suites following defect identification or feature modifications, executing them before any refactoring processes.
• Communicate coverage metrics, known limitations, and release risks, while providing support to developers with shared tools and fixtures.
• Over 5 years of experience in QA automation or as a Software Development Engineer in Test (SDET).
• Proficient in test automation using TypeScript and/or Python.
• Experienced in API and integration testing; familiarity with Playwright or similar tools for UI automation.
• Proven track record of integrating automated tests into CI pipelines, with effective management of test data and fixtures.
• Practical experience using AI tools for test generation, coupled with a critical assessment of the generated outputs.
• Ability to make risk-based decisions regarding what requires thorough testing and what can be less rigorously tested in prototypes.
• Proficient in English.
• Capable of working core hours based on US Central Time.
• Experience in evaluating large language models (LLM) or other AI outputs, including evaluation sets, scoring, and regression assessments.
• Knowledge of testing document outputs such as PDFs, spreadsheets, or generated reports.
• Familiarity with engineering, construction, or CAD/BIM data.
• Experience with applications hosted on AWS.
• Understanding of engineering codes or domains related to safety and regulations.
• Outstanding work environment recognized by Great Place To Work certification.
• A multicultural organization that is continually expanding.
CACI International Inc
Helpware
mpathic
Leidos
Get handpicked remote jobs straight to your inbox weekly.