Senior Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Build and Sustain Bronze/Silver Layer Pipelines: You will ensure that core data sources are accurately ingested, timely delivered, and fully traceable.

• Oversee Data Ingestion, Transformation, and Enrichment: You will manage the complete pipeline from the arrival of raw files through to cleansed, standardized staging tables, including deduplication, standardization, code mapping, and entity resolution.

• Create Automated Ingestion Pipelines: You will utilize Snowpipe, Matillion, or bespoke solutions that prioritize reliability, observability, and minimal manual effort.

• Implement dbt Models: You will produce clean, tested, and documented SQL that adheres to medallion architecture patterns and facilitates incremental processing at scale.

• Develop Scalable Engineering Patterns: You will design reusable templates, macros, and testing frameworks that expedite the onboarding of new data sources while maintaining high quality.

• Guarantee Data Quality and Observability: You will execute dbt tests, freshness checks, and anomaly detection, taking ownership of pipeline reliability and addressing data quality incidents as they arise.

• Collaborate with the Architect and Stakeholders: You will convert business requirements into technical specifications, engage in design reviews, contribute to ADRs, and ensure that pipelines comply with enterprise standards.

⛳️ Requirements

• Over 5 years of professional data engineering experience, with a minimum of 3 years in contemporary cloud data environments.

• More than 2 years of hands-on experience with Snowflake, encompassing performance tuning, Snowpipe, and Snowflake-specific optimization techniques.

• At least 2 years of working with dbt (Core or Cloud) in live environments, including incremental models, testing, documentation, and package management.

• Proven experience with healthcare data, including familiarity with ICD-10, NPI, NDC codes, and the complexities surrounding claims data quality.

• Production experience with orchestration tools like Airflow, Dagster, or similar, having built and maintained DAGs that function reliably at scale.

• Strong SQL capabilities, with the ability to compose efficient, readable, well-structured SQL and troubleshoot complex query performance issues.

• Proficient in Python for data processing, scripting, and automating pipelines.

🏝️ Benefits

• Location and Travel: This position is remote-friendly, allowing for flexibility for candidates based in the EU or India.

• Occasional travel for stakeholder meetings and strategic planning sessions.