GCP Data Engineering Manager

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the comprehensive migration of enterprise data platforms from AWS to GCP.

• Evaluate the AWS architecture, data pipelines, workloads, dependencies, and operational procedures.

• Establish the target-state GCP architecture, migration roadmap, phases, dependencies, risks, and rollback strategies.

• Facilitate architecture reviews and technical design discussions.

• Design and implement scalable GCP Data Lake and Lakehouse platforms utilizing Google Cloud Storage and BigQuery.

• Define Bronze, Silver, and Gold/Atomic data layers along with scalable frameworks for ingestion, transformation, and consumption.

• Analyze and enhance AWS data platforms, mapping AWS services to equivalent or superior GCP services.

• Architect real-time data pipelines using Pub/Sub, Dataflow/Apache Beam, BigQuery, and Cloud Storage.

• Create designs for high-volume event ingestion, enrichment, transformation, monitoring, logging, and alerting.

• Design and execute batch and real-time ETL/ELT pipelines using Python, PySpark, SQL, Dataflow/Apache Beam, BigQuery, and dbt.

• Devise CDC and real-time ingestion patterns while orchestrating workflows with Cloud Composer/Airflow.

• Create enterprise analytics and reporting data models, which include dimensional, normalized, denormalized, and multi-tenant models.

• Optimize BigQuery features like partitioning, clustering, SQL, query execution, Dataflow, Spark, Cloud Storage, and streaming workloads.

• Establish standards for data governance, quality, lineage, metadata, ownership, security, and compliance.

• Implement IAM, least-privilege access, encryption, service accounts, network security, and data access policies.

• Lead Terraform infrastructure automation, CI/CD, testing, deployment, and provisioning across Development, QA, UAT, and Production environments.

• Develop FinOps, performance optimization, cost optimization strategies, benchmarks, and SLAs.

• Guide, mentor, and technically support Data Engineers, Senior Data Engineers, and Technical Leads.

• Conduct architecture and code reviews, establish roadmaps and priorities, and track progress, risks, dependencies, and milestones.

• Serve as the primary technical liaison for US-based stakeholders and collaborate with Business, Product, Data Science, BI, and DevOps teams.

• Translate requirements into scalable solutions and communicate architecture decisions, migration strategies, roadmaps, risks, timelines, and trade-offs.

⛳️ Requirements

• Over 15 years of experience in Data Engineering, Data Architecture, or Cloud Engineering.

• More than 5 years of hands-on experience in GCP Data Engineering.

• Strong practical experience with AWS Data Engineering and Architecture.

• Demonstrated experience in AWS-to-GCP migration projects.

• Significant experience in designing enterprise Data Lake / Lakehouse platforms.

• Proven experience in migrating AWS data workloads to GCP.

• In-depth knowledge of AWS and GCP service mapping and cloud migration methodologies.

• Expert-level SQL skills and strong proficiency in Python/PySpark.

• Solid background in data modeling and data warehousing.

• Experience with Terraform and CI/CD processes.

• Proven experience in managing and mentoring data engineering teams.

• Excellent communication skills with experience engaging with US-based stakeholders.

• Capability to assess AWS workloads, define the GCP target architecture, and lead migration through to production implementation.

• Strong familiarity with AWS services including S3, Glue, Glue Data Quality, Redshift/Redshift Serverless, Athena, Step Functions, DMS, Lake Formation, and IAM.

• Comprehensive knowledge of GCP services such as BigQuery, Google Cloud Storage, Pub/Sub, Dataflow/Apache Beam, Cloud Composer/Airflow, Dataproc/Spark, Cloud Monitoring, Cloud Logging, and Dataplex/Data Catalog.

• Experience with ETL/ELT, CDC, batch and streaming data processing, event-driven architecture, and data pipeline optimization.

• Expertise in enterprise Data Lake/Lakehouse, Medallion Architecture, data modeling, dimensional modeling, multi-tenant data modeling, schema-on-read/schema-on-write, data lineage, metadata management, and data governance.

• Familiarity with dbt, Apache Airflow, Terraform, Git/GitHub, Cloud Build, CI/CD, and data quality frameworks.

• OpenLineage experience is a plus.

• Experience as an Engineering Manager on the GCP platform; current Senior/Lead Data Engineer profiles are not suitable.

🏝️ Benefits

• Flexible remote work environment.

• Exposure to global enterprise customers.

• Collaborative, innovation-driven engineering culture.

• Continuous learning and certification opportunities.

• Opportunity to lead large-scale AWS-to-GCP cloud transformation initiatives.

• Work on enterprise Data Lakehouse and analytics modernization projects.