Automation Test Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Assist in the development of an AI/ML platform that utilizes generative AI to alleviate user challenges within the Centers for Medicare and Medicaid Services (CMS)

• Create and sustain scalable automated testing frameworks for web applications, APIs, and AI-centric functionalities

• Construct automated pipelines for model validation, bias detection, and monitoring drift in machine learning systems

• Define and carry out functional, regression, performance, and AI-specific testing strategies

• Collaborate with AI/ML engineers to generate test datasets and emulate real-world user interactions and edge scenarios

• Establish quality gates integrated with CI/CD for reliable AI releases and fairness metrics

• Guide junior engineers and promote best practices in intelligent automation and AI testing

• Champion ethical AI standards and engage in discussions surrounding responsible AI

⛳️ Requirements

• Over 5 years of experience in software QA/automation engineering, with a minimum of 1 year dedicated to AI/ML product testing

• Strong expertise in at least one automation framework (e.g., Selenium, Cypress, Playwright) and a scripting language (Python is preferred)

• Practical experience in testing AI models, including validation metrics, dataset versioning, and model interpretability

• Familiarity with tools such as MLflow, TensorBoard, or similar platforms for AI observability

• Knowledge of model bias, fairness audits, and regulatory compliance considerations in government or regulated settings

• Experience in Agile DevSecOps environments utilizing modern CI/CD tools (Jenkins, GitHub Actions, GitLab, etc.)

• Capability to obtain and maintain a Public Trust security clearance

• Acceptable results from a background investigation

• Excellent collaboration abilities with effective communication among engineering, data science, and product teams

🏝️ Benefits

• Medical, dental, and vision insurance

• 15 days of paid leave

• 7 days of sick leave

• 2 days of bereavement leave

• 11 paid Federal holidays

• Up to 40 hours of jury duty leave

• 401K plan with a 4% employer contribution (with no vesting period)

• Up to 4 weeks of paid paternity and maternity leave

• Company-provided laptop

• $5,000 annually for professional development

• $600 annually for technical supplies and equipment

• $2,000 referral bonus

• Life and disability insurance

• HSA and FSA options

• Voluntary Legal Shield and ID Shield benefits

• Flexible working hours

• Remote work opportunities

• Chance to be part of a collaborative, driven team focused on modernizing government services through innovative technology and solutions