Senior Automation Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Assist in the development, validation, and automation of an AI-driven Patient Health Data Summarization feature within the JLV Clinical Decision Support platform.

• Create detailed test plans, test cases, and acceptance criteria.

• Establish traceability between business requirements, clinical data standards, and testing scenarios.

• Design testing coverage for CCDs, Discharge Summaries, Progress Notes, and History & Physical documents.

• Devise positive, negative, boundary, and edge-case testing scenarios.

• Analyze and validate C-CDA XML documents, focusing on headers, sections, templates, and clinical entries.

• Confirm the extraction and summarization of medications, allergies, laboratory results, procedures, care plans, and diagnoses.

• Ensure that AI-generated summaries accurately represent source documentation and maintain traceability to clinical sources.

• Conduct functional, integration, system, regression, performance, and user acceptance testing.

• Develop and uphold automated test suites for document ingestion, AI summarization, APIs, and user interfaces.

• Test CDS workflows and scenarios driven by patient context across various data sources and encounters.

• Support CI/CD pipelines through automated quality gates and continuous testing.

• Construct automated validation frameworks for AI-generated summaries.

• Assess AI-generated content for completeness, consistency, usability, accuracy, and reliability.

• Identify omissions, inaccuracies, hallucinations, and other defects in AI-generated outputs.

• Validate AI guardrails, monitoring, auditability, and quality controls.

• Engage in defect triage, root-cause analysis, and release-readiness activities.

• Produce test documentation, defect reports, traceability matrices, and quality assessments.

• Collaborate with software engineers, solution architects, clinicians, product owners, cybersecurity teams, and government stakeholders.

• Utilize AI-assisted tools to enhance testing, automation, regression testing, defect analysis, and quality assurance.

• Undertake additional tasks as necessary to support team objectives and project success.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Health Informatics, Engineering, or a related technical field, or equivalent experience.

• At least 5 years of experience in testing healthcare software applications.

• Strong understanding of Consolidated Clinical Document Architecture (C-CDA) standards, including the structure, sections, templates, and entries of clinical documents.

• Practical experience in reading, validating, and troubleshooting XML-based healthcare data.

• Familiarity with healthcare interoperability standards and clinical data exchange.

• Experience testing Clinical Decision Support (CDS) applications and workflows.

• Knowledge of SMART on FHIR, CDS Hooks, REST APIs, and related interoperability technologies.

• Proficiency with test automation frameworks, API testing tools, and defect management platforms.

• Experience in developing and maintaining automated testing solutions within Agile and DevSecOps environments.

• Familiarity with AI-assisted testing tools and techniques for test case generation, automation development, regression testing, defect analysis, and quality assurance.

• Knowledge of Generative AI and Large Language Models (LLMs), including their use in creating and maintaining automated test scripts, validation frameworks, and test data.

• Experience in validating AI-generated outputs and implementing quality controls for accuracy, consistency, repeatability, and reliability.

• Strong analytical, troubleshooting, and problem-solving abilities.

• Excellent written and verbal communication skills.

• Additional qualifications may include experience with VA healthcare systems, knowledge of USCDI, familiarity with SNOMED CT/LOINC/RxNorm/ICD, AI/ML healthcare solution validation, AI-powered testing platform experience, autonomous or semi-autonomous AI testing workflows, prompt engineering, AI evaluation methodologies, accessibility and usability testing, Agile/DevSecOps/cloud-native experience, and exposure to federal healthcare modernization.

• U.S. Citizenship is required.

• All selected candidates must successfully pass a background check.

• Ability to obtain and maintain a Government/Public Trust clearance, including necessary fingerprinting, when required.

• Sponsorship is not available.

🏝️ Benefits

• Medical/Dental/Vision coverage.

• 401k plan with Employer Match.

• Paid Time Off plus Federal Holidays.

• Corporate Laptop provided.

• Opportunities for Training.

• Remote work opportunity.