Senior Data Engineer

This is a fully remote position, open to applicants in California, +11 more states.

📋 Description

• Take ownership of the target architecture for the warehouse and curated layer, documenting design decisions and guiding future architectural direction.

• Create a semantic layer that serves as a centralized metric repository, complete with definitions, formulas, and reasoning.

• Set standards for naming conventions, labeling, data lineage, enumerated values, and the history of business rules.

• Develop a configuration model tailored for practice-specific business rules.

• Collaborate daily with data engineers to review their work.

• Construct the gold layer and a baseline report library for new practices, addressing patient journeys, practice capacity, and growth metrics.

• Expand infrastructure as code and continuous integration across the warehouse and pipelines, including schema-drift tests.

• Organize unstructured clinical records, including HTML and PDF journals, notes, and form submissions.

• Streamline migration profiling, delta reconciliation, data quality assessments, and report provisioning through automation.

• Implement AI within pipelines for enrichment, anomaly detection, metadata generation, data dictionaries, and enumerated catalogs.

• Ensure the warehouse is secure and functional for AI applications by cataloging, enabling semantic queries, creating evaluation sets, and governing patient-data tools.

• Design repeatable migrations featuring profiling, delta matching, schema mapping, record matching, and reporting on rejected records.

• Collaborate with engineering teams on platform modifications and warehouse-platform change agreements.

• Partner with product and commercial leadership to establish meaningful business metrics.

⛳️ Requirements

• Over 8 years of experience in data engineering, encompassing end-to-end ownership of a warehouse or lakehouse architecture, from data ingestion to the curated layer utilized for querying.

• Proficient in SQL and Python, with hands-on production experience in a modern cloud warehouse environment.

• Familiarity with platforms such as Redshift, Snowflake, BigQuery, or Databricks.

• Practical experience in developing a semantic or metrics layer using dbt or similar tools.

• Production-level experience with orchestration and infrastructure as code on AWS, including tools like Airflow and Terraform.

• Experience in designing and maintaining dimensional or Data Vault models in a production setting.

• Skilled in data quality testing and observability, utilizing dbt tests, Great Expectations, or comparable tools.

• Regular use of coding agents and AI tools in personal projects.

• Experience with AI applications within a data platform, such as LLM classification of unstructured records, entity matching, anomaly detection, or natural language querying over a semantic layer, including evaluation sets.

• Knowledge of regulated data, particularly in healthcare or financial sectors.

• Must possess current and valid authorization to work in the country of application.

• Adoreal does not provide visa sponsorship for this position.

🏝️ Benefits

• Comprehensive healthcare coverage for you and your family.

• 401k retirement plan.

• Paid time off (PTO).

• Paid holidays.

• Opportunities for company equity.

• Fully remote work environment with flexible scheduling options.

• A collaborative and thriving team culture.

• Opportunities for promotions and professional growth.

• Disability accommodations available throughout the recruitment process.