Senior Data Engineer – GCP, Python, Iceberg, Delta Lake, Kafka, Snowflake, Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and develop the UniForm write layer utilizing both Delta and Iceberg dual metadata.

• Create ingestion pipelines from GCS to BigQuery for structured operational datasets.

• Develop and implement solutions using Python and Spark.

• Establish Kafka-based CDC patterns for real-time and near-real-time data ingestion.

• Create data lineage, dependency tracking, and modular adapter code.

• Configure external tables in Snowflake Horizon for zero-copy reads.

• Write data hub tables with Delta Lake enabled for Iceberg UniForm to ensure consumption by target platforms without conversion.

• Implement and validate Delta Sharing endpoints for users of Databricks.

• Construct governed access layer components, including RBAC, connector registry entries, and tenant-scoped authorization.

• Align semantic layer models with LookML and definitions in the KPI catalog.

• Collaborate with cross-functional teams to deliver comprehensive end-to-end features.

• Troubleshoot issues across the entire stack and enhance code quality.

• Design reusable data-sharing adapters that connect BigQuery-backed data products to Snowflake and Databricks.

⛳️ Requirements

• Over 6 years of demonstrated enterprise-level experience in Python and Spark.

• Advanced expertise in GCP BigQuery.

• Strong working knowledge of Apache Iceberg, Delta Lake, and Iceberg UniForm.

• Experience with Delta Sharing and Kafka / CDC pipeline implementations.

• Specific work experience with Snowflake Horizon Catalog and Databricks Unity Catalog.

• Solid background in data lake architecture and the design of ingestion pipelines.

• Excellent communication abilities and a strong aptitude for collaboration with multiple teams.

• Previous experience in enterprise SaaS, media, or advertising technology is a plus.

• Regular daily use of AI-assisted development tools; familiarity with Claude Code is preferred.

🏝️ Benefits

• Medical insurance

• Dental insurance

• Vision insurance

• Pet insurance

• Paid Time Off

• 401(k)

• Competitive compensation

• 100% remote work

• Full-time employment