Principal Data Platform Engineer, Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the complete architecture and delivery of scalable data solutions, focusing on Databricks-based platforms and contemporary cloud ecosystems.

• Spearhead the design and execution of data pipelines, data models, and transformation frameworks that facilitate analytics, reporting, and advanced use cases.

• Act as the primary technical leader facing clients, fostering trusted relationships and assisting stakeholders in navigating complex data challenges and solution decisions.

• Convert vague business requirements into clear, actionable technical architectures and delivery strategies.

• Establish and uphold best practices across data engineering, encompassing data ingestion (pipeline orchestration, testing, and optimization) and DevOps.

• Drive decisions regarding platform strategy and architecture, including lakehouse design, medallion architecture, and governance frameworks.

• Mentor and lead delivery teams, offering technical guidance, conducting code reviews, and providing hands-on support to achieve successful project outcomes.

• Collaborate with cross-functional teams, including data architects, analysts, and client stakeholders, to ensure alignment and value delivery.

• Identify risks, proactively address challenges, and ensure high-quality, timely delivery across projects.

• Contribute to internal capability building, including reusable frameworks, accelerators, and thought leadership initiatives.

• Support business development efforts by shaping technical solutions, contributing to proposals, and engaging in client discussions.

⛳️ Requirements

• Over 7 years of experience in data engineering, demonstrating a trajectory into technical leadership and architecture ownership.

• Extensive expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing.

• Advanced skills in SQL and Python, with significant experience in building and optimizing large-scale data pipelines.

• Practical experience with cloud platforms such as AWS, Azure, or GCP, including data services and infrastructure design.

• Strong comprehension of data modeling principles, ETL/ELT patterns, and distributed data processing.

• Familiarity with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt).

• Proven capability to lead technical delivery while remaining hands-on as necessary.

• Strong client-facing experience, including gathering requirements, designing solutions, and communicating with executives.

• Ability to navigate ambiguity, prioritize effectively, and clarify complex environments.

• Healthcare data experience (e.g., Epic, HL7, FHIR, claims data) is highly preferred.

• Experience with CI/CD, DevOps practices, and infrastructure-as-code tools is a plus.

🏝️ Benefits

• Competitive salaried and hybrid compensation plans.

• Health care plan (medical, HSAs, dental & vision).

• Retirement plan (401k).

• Life insurance (basic, voluntary & AD&D).

• Dependent & health savings accounts.

• Short-term & long-term disability.

• Paid time off (vacation/sick & public holidays).

• Training & development fund.

• Work from home flexibility.

• Charitable contributions to causes you believe in.