Senior DevOps Automation Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop detailed test plans, cases, and acceptance criteria for AI-driven patient summarization features.

• Establish traceability between business requirements, clinical data standards, and testing scenarios.

• Design test coverage for various clinical document types, including CCDs, Discharge Summaries, Progress Notes, and History & Physical documents.

• Create positive, negative, boundary, and edge-case testing scenarios for AI-enabled clinical applications.

• Analyze and validate C-CDA XML documents, focusing on headers, sections, templates, and clinical entries.

• Verify the extraction and summarization of medications, allergies, laboratory results, procedures, care plans, and diagnoses.

• Ensure that AI-generated summaries accurately reflect the source clinical documentation while maintaining traceability to the original sources.

• Conduct functional, integration, system, regression, performance, and user acceptance testing.

• Develop and maintain automated test suites for document ingestion, AI summarization, APIs, and user interfaces.

• Test Clinical Decision Support (CDS) workflows and patient-context-driven launch scenarios across multiple data sources and patient encounters.

• Support CI/CD pipelines via automated quality gates and continuous testing.

• Build automated validation frameworks for AI-generated summaries.

• Assess AI-generated content for clinical completeness, consistency, usability, accuracy, and reliability.

• Identify omissions, inaccuracies, hallucinations, and other AI-generated defects that impact clinical workflows.

• Validate AI guardrails, monitoring capabilities, auditability, and quality controls.

• Engage in defect triage, root-cause analysis, and release-readiness activities.

• Create test documentation, defect reports, traceability matrices, and quality assessments.

• Collaborate with software engineers, solution architects, clinicians, product owners, cybersecurity teams, and government stakeholders.

• Utilize AI-assisted tools to expedite testing, automation, regression testing, defect analysis, and quality assurance.

• Undertake additional responsibilities as necessary to support team objectives and project success.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Health Informatics, Engineering, or a related technical field, or equivalent experience.

• At least 5 years of experience in testing healthcare software applications.

• Strong understanding of Consolidated Clinical Document Architecture (C-CDA) standards, including document structure, sections, templates, and entries.

• Practical experience in reading, validating, and troubleshooting XML-based healthcare data.

• Familiarity with healthcare interoperability standards and clinical data exchange.

• Experience in testing Clinical Decision Support (CDS) applications and workflows.

• Knowledge of SMART on FHIR, CDS Hooks, REST APIs, and related interoperability technologies.

• Experience with test automation frameworks, API testing tools, and defect management systems.

• Proven experience in developing and maintaining automated testing solutions within Agile and DevSecOps environments.

• Familiarity with AI-assisted testing tools and techniques for test case generation, automation development, regression testing, defect analysis, and quality assurance.

• Understanding of Generative AI and Large Language Models (LLMs), including the use of AI for creating and maintaining automated test scripts, validation frameworks, and test data.

• Experience in validating AI-generated outputs and establishing quality controls for accuracy, consistency, repeatability, and reliability.

• Strong analytical, troubleshooting, and problem-solving capabilities.

• Excellent written and verbal communication skills.

• Additional qualifications may include experience with VA healthcare systems, knowledge of USCDI, healthcare terminology, AI/ML healthcare solution validation, AI-powered testing platforms, autonomous or semi-autonomous AI testing workflows, prompt engineering, accessibility and usability testing, Agile/DevSecOps/cloud-native environments, and federal healthcare modernization exposure.

• U.S. Citizenship is required.

• Must successfully pass a background check.

• Ability to obtain and maintain a Government/Public Trust clearance, including required fingerprinting, when necessary.

• Sponsorship is not provided.

🏝️ Benefits

• Medical/Dental/Vision coverage.

• 401k plan with Employer Match.

• Paid Time Off (PTO) plus Federal Holidays.

• Corporate laptop provided.

• Opportunities for training.

• Remote work opportunity.