Senior Test Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Create detailed test plans, test cases, and acceptance criteria for AI-driven patient summarization functionalities.

• Establish traceability among business requirements, clinical data standards, and testing scenarios.

• Design test coverage for various clinical document types, including CCDs, Discharge Summaries, Progress Notes, and History & Physical documents.

• Develop testing scenarios for positive, negative, boundary, and edge cases in AI-enabled clinical applications.

• Analyze and validate C-CDA XML documents, focusing on headers, sections, templates, and clinical entries.

• Ensure precise extraction and summarization of clinical information, such as medications, allergies, laboratory results, procedures, care plans, and diagnoses.

• Confirm that AI-generated summaries accurately represent source clinical documentation and maintain traceability to original clinical sources.

• Conduct functional, integration, system, regression, performance, and user acceptance testing.

• Create and sustain automated test suites for document ingestion, AI summarization, APIs, and user interfaces.

• Test CDS workflows and patient-context-driven launch scenarios across various data sources and patient encounters.

• Facilitate CI/CD pipelines through automated quality gates and continuous testing.

• Develop automated validation frameworks for AI-generated summaries.

• Assess AI-generated content for clinical completeness, consistency, usability, accuracy, and reliability.

• Identify omissions, inaccuracies, hallucinations, and other AI-related defects that might affect clinical workflows.

• Validate AI guardrails, monitoring capabilities, auditability, and quality controls.

• Engage in defect triage, root-cause analysis, and release-readiness efforts.

• Produce test documentation, defect reports, traceability matrices, and quality assessments.

• Collaborate with software engineers, solution architects, clinicians, product owners, cybersecurity teams, and government stakeholders.

• Utilize AI-assisted tools to enhance test development, test automation, regression testing, defect analysis, and quality assurance tasks.

• Undertake additional duties and responsibilities as required to support team goals and ensure project success.

⛳️ Requirements

• A Bachelor's degree in Computer Science, Information Technology, Health Informatics, Engineering, or a related technical field, or equivalent experience.

• A minimum of 5 years of experience in testing healthcare software applications.

• Strong understanding of Consolidated Clinical Document Architecture (C-CDA) standards, including clinical document structure, sections, templates, and entries.

• Practical experience in reading, validating, and troubleshooting XML-based healthcare data.

• Familiarity with healthcare interoperability standards and clinical data exchange.

• Experience in testing Clinical Decision Support (CDS) applications and workflows.

• Knowledge of SMART on FHIR, CDS Hooks, REST APIs, and related interoperability technologies.

• Proficient in test automation frameworks, API testing tools, and defect management platforms.

• Experience in developing and maintaining automated testing solutions in Agile and DevSecOps environments.

• Familiarity with AI-assisted testing tools and techniques for test case generation, automation development, regression testing, defect analysis, and quality assurance.

• Understanding of Generative AI and Large Language Models (LLMs), including leveraging AI for creating and maintaining automated test scripts, validation frameworks, and test data.

• Experience in validating AI-generated outputs and establishing quality controls for accuracy, consistency, repeatability, and reliability.

• Strong analytical, troubleshooting, and problem-solving capabilities.

• Excellent written and verbal communication skills.

• Experience supporting Department of Veterans Affairs (VA) healthcare systems, including JLV or similar clinical viewer applications.

• Knowledge of USCDI and healthcare interoperability requirements.

• Familiar with SNOMED CT, LOINC, RxNorm, and ICD.

• Experience in validating AI/ML-enabled healthcare solutions.

• Familiarity with AI-powered testing platforms such as GitHub Copilot, Microsoft Copilot, Testim, Functionize, Mabl, or similar tools.

• Experience in creating autonomous or semi-autonomous AI testing workflows.

• Understanding of prompt engineering and AI evaluation methodologies within software quality assurance.

• Knowledge of accessibility and usability testing in healthcare settings.

• Experience in Agile, DevSecOps, and cloud-native development environments.

• Exposure to healthcare data analytics and modernization initiatives within federal healthcare agencies.

• U.S. Citizenship is required.

• All selected candidates must successfully complete a background check.

• Ability to obtain and maintain a Government/Public Trust clearance, including necessary fingerprinting, when required for the position.

• Sponsorship is not available.

🏝️ Benefits

• Medical/Dental/Vision coverage.

• 401k plan with Employer Match.

• Paid Time Off (PTO) plus Federal Holidays.

• Corporate Laptop provided.

• Training Opportunities available.

• Remote work opportunity.