Databricks Data Engineer, SME

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the technical design and execution of the RevOS data architecture within Databricks and the DHA War Data Platform.

• Evaluate and enhance existing DHA revenue-cycle data structures, focusing on schemas, entities, definitions, relationships, keys, data quality, and reconciliation.

• Create a canonical revenue-cycle data model that integrates patient, encounter, authorization, provider, documentation, diagnosis, procedure, coding, charge, claim, payment, denial, accounts receivable, appeal, follow-up, and recovery entities.

• Design and implement the Databricks Bronze → Silver → Gold architecture for RevOS.

• Develop Gold data products and analytical data marts for dashboard and visualization workloads.

• Create and sustain Databricks SQL dashboards, AI/BI dashboards, and native visualizations.

• Construct reusable semantic datasets and optimized SQL structures for RevOS KPI calculations across enterprise, DHN, MTF, department, provider, encounter, and claim levels.

• Enhance dashboard queries and Gold datasets for performance, refresh cadence, scalability, and concurrent users.

• Configure dashboard access utilizing Unity Catalog permissions, row-level security, column-level security, PHI/PII requirements, and user roles.

• Facilitate dashboard drill-down from Healthy / At Risk / Critical indicators into record-level exceptions and work queues.

• Profile Bronze datasets and develop source-to-target mappings.

• Normalize and conform data from MHS GENESIS Millennium, Abacus, and other sanctioned sources.

• Create reusable Silver-layer entities with common keys, normalized timestamps, reference dimensions, and standardized definitions.

• Build Gold products that support revenue-cycle KPIs, coding-audit impact, charge completeness, claims readiness, Days-to-Bill, denial management, payer performance, payment/remittance reconciliation, underpayment detection, AR aging, revenue recovery, and audit/NFR traceability.

• Design the data model that underpins the Revenue Opportunity Ledger and recovery work queues.

• Establish entity-resolution methodologies and deterministic/probabilistic matching across source systems.

• Develop and maintain ODCS-based machine-readable data contracts.

• Implement automated quality controls and quality gates for Bronze-to-Silver and Silver-to-Gold transitions.

• Construct reconciliation controls across billed, allowed, paid, adjusted, patient-responsibility, AR, and recovery values.

• Configure and manage the Databricks Unity Catalog, including catalog/schema/table design, classification, tagging, ownership, lineage, security, and access controls.

• Assist with enterprise metadata federation and DHA data-governance requirements.

• Create scalable Delta Lake structures and optimize partitioning, clustering, SQL performance, storage, and compute usage.

• Develop Databricks jobs, workflows, orchestration, production monitoring, alerting, and error handling.

• Version-control code, transformations, contracts, configurations, and infrastructure-as-code using Government-furnished GitLab.

• Facilitate automated CI/CD and controlled production promotion.

• Collaborate with Data Scientists, Analytics Engineers, Revenue Cycle, and Coding SMEs to support modeling, analytics, recovery scoring, anomaly detection, dashboards, and business rules.

• Create source-to-target documentation, data dictionaries, lineage artifacts, architecture diagrams, runbooks, and sustainment documentation.

• Mentor Data Engineers and create reusable Databricks engineering and visualization patterns.

⛳️ Requirements

• Active SECRET Clearance.

• Bachelor's Degree and 10+ years of experience in enterprise data engineering, data architecture, data platform development, or related fields.

• Senior/SME-level hands-on experience with Databricks.

• Proven experience utilizing Databricks visualization and dashboard capabilities, including Databricks SQL and/or AI/BI dashboards.

• Ability to design foundational Gold/semantic data structures for enterprise-scale dashboards and operational reporting.

• Experience in crafting and optimizing SQL queries, datasets, and views that support interactive visualization and drill-down.

• Strong background with Apache Spark / PySpark, SQL, Python, Delta Lake, ETL / ELT, data pipeline orchestration, and large-scale data transformation.

• Demonstrated experience in implementing Bronze / Silver / Gold medallion architectures.

• Significant experience in designing normalized and analytical data models for complex enterprise environments.

• Background in evaluating and redesigning or fixing poorly structured existing data environments.

• Experience in designing canonical data models, conformed dimensions, enterprise keys, entity resolution, and reusable semantic structures.

• Solid understanding of data-quality engineering, financial reconciliation, metadata, lineage, and data governance.

• Experience with Databricks Unity Catalog or similar enterprise data-governance/catalog technology.

• Experience in building production-grade pipelines with automated testing, monitoring, logging, failure handling, and CI/CD.

• Capacity to translate operational workflows into logical and physical data architectures and visualization-ready products.

• Experience supporting highly regulated environments in healthcare, finance, or Government.

• Ability to handle PHI, PII, CUI, and other controlled data under relevant security and privacy regulations.

• Competence in collaborating across engineering, analytics, visualization, cybersecurity, product, architecture, and business-SME teams.

• Capability to meet applicable DHA/DoD security, privacy, access, and data-handling criteria.

• Preferred: Previous experience with Advana and/or the current War Data Platform (WDP).

• Preferred: Direct experience in developing Databricks data products and dashboards within a DoD enterprise environment.

• Preferred: Experience with DHA, MHS, or other DoD healthcare data.

• Preferred: Experience with MHS GENESIS / Oracle Health/Cerner Millennium and/or Abacus.

• Preferred: Background in healthcare revenue-cycle data.

• Preferred: Familiarity with healthcare EDI transactions including 837, 835, 270/271, 276/277, and 278.

• Preferred: Experience in implementing ODCS or equivalent machine-readable contracts.

• Preferred: Experience with GitLab-based DevSecOps, infrastructure-as-code, automated testing, and secure promotion.

• Preferred: Familiarity with Collibra or similar enterprise data catalogs.

• Preferred: Experience supporting financial auditability, reconciliation, lineage, and audit-remediation initiatives.

• Preferred: Familiarity with DoD RMF, NIST, IL4/IL5 environments, and federal data-governance requirements.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health, dental, and vision insurance.

• Generous paid time off and holidays.

• Retirement savings plan with company matching.

• Opportunities for professional development and continued education.