Principal Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the long-term technical architecture for DPE, focusing on ingestion, orchestration, event streaming, and the underlying platform infrastructure that supports transformation and serving, influencing critical decisions for the CDC-based streaming platform (Kafka → Flink → BigQuery), orchestration platform evaluation, and lower environment strategy.

• Lead the Architecture Review Committee (ARC) decisions and serve as the primary technical DRI for cross-team, multi-system, and cost-impacting changes.

• Establish and enforce engineering standards and production readiness criteria for all DPE-owned systems, including testing requirements, CI/CD patterns, observability-as-code, logging standards, data contracts, Schema Registry governance, and defining what 'production-ready' means for emerging streaming and CDC capabilities.

• Oversee data quality and observability architecture, which includes dbt anomaly detection frameworks, schema validation, data drift alerting, and platform standards that ensure data integrity for consumers.

• Define the technical strategy for self-service analytics: identifying platform capabilities that allow Analytics Engineering to operate independently, establishing guardrails to prevent downstream issues, and outlining how DPE mitigates bottlenecks over time.

• Manage the evaluation, onboarding, and continual governance of DPE-managed tools such as Fivetran and Confluent, including contract management, cost tracking, and decisions regarding deprecation.

• Oversee data sharing and egress patterns, including access provisioning, cross-team data contracts, reverse ETL (Hightouch), and regulated consumption paths for both internal and external consumers.

• Drive cost governance for platform infrastructure, focusing on BigQuery slot reservations, query optimization, partition strategies, orchestration rightsizing, and ensuring accountability for cloud spending across the entire DPE stack.

• Lead incident response efforts for platform-level P1/P2 incidents, acting as a technical escalation point, facilitating blameless RCAs, and implementing systemic fixes to prevent recurrence.

• Produce high-quality technical documentation, such as architecture decision records, solution design documents, and RFCs, that foster alignment and serve as the team's reference standard.

• Mentor and develop Staff and Senior Data Engineers, enhancing the technical capabilities through design reviews, code reviews, and hands-on collaboration.

• Collaborate cross-functionally with ML/Data Science, legal/security/compliance, and DevOps to deliver platform capabilities that are ML-ready, compliant with HIPAA/GDPR, and robust at the infrastructure level.

• Engage directly in critical path work.

⛳️ Requirements

• A minimum of 15 years of professional experience in designing, building, and managing data platform architecture at a company-wide scale.

• Proven ability to define and align architectural vision with business objectives across an entire engineering organization.

• Extensive expertise in cloud-native data platforms, particularly within GCP (BigQuery, GCS, Dataflow); familiarity with AWS operational environments (EKS-based Airflow) is required, with a strong preference for BigQuery; fluency in multi-cloud environments is essential.

• Hands-on experience with the modern data stack, including governance of dbt at platform scale, Airflow/Astronomer, Kafka/Confluent, Databricks/Spark, Fivetran, and data sharing/activation platforms (Hightouch or similar).

• Experience in designing and operating event streaming pipelines at scale, including Schema Registry, data contracts, and managing consumer lag.

• A demonstrated record of establishing engineering standards across multiple teams and successfully driving their adoption without direct authority.

• Experience managing data quality frameworks, including dbt testing, anomaly detection, schema validation, and data observability tools.

• Familiarity with data governance and compliance frameworks within a regulated environment, including HIPAA/PHI handling, data classification, access controls, audit logging, and GDPR compliance.

• Experience leading incident response for data platform outages, including conducting blameless RCAs, identifying systemic root causes, and implementing operational improvements.

• Proficiency in infrastructure-as-code practices, particularly with Terraform or equivalent tools; you approach infrastructure changes the same way as software modifications.

• Strong skills in Python and SQL, with the ability to write, review, and enhance production-grade pipeline code.

• Excellent written communication skills; you create design documents and RFCs that foster clarity and alignment, not confusion. You are comfortable operating in ambiguous situations, taking initiative to define the path forward.

🏝️ Benefits

• Competitive salary and equity compensation for full-time positions.

• Unlimited PTO, company holidays, and quarterly mental health days.

• Comprehensive health benefits, including medical, dental, and vision coverage, along with parental leave.

• Employee Stock Purchase Program (ESPP).

• 401k benefits with employer matching contributions.

• Offsite team retreats.