Data Engineering Manager – AWS to GCP Migration

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the comprehensive migration of enterprise data platforms from AWS to GCP.

• Evaluate the AWS architecture, data pipelines, workloads, dependencies, and operational workflows.

• Create the target-state GCP architecture, migration roadmap, phases, risks, dependencies, and rollback strategies.

• Design and implement scalable GCP Data Lake and Lakehouse platforms utilizing Google Cloud Storage and BigQuery.

• Develop frameworks for batch and real-time ETL/ELT, CDC, streaming ingestion, transformation, orchestration, and consumption.

• Facilitate architecture reviews, technical design discussions, code evaluations, and set engineering standards.

• Establish standards for data modeling, partitioning, clustering, governance, lineage, metadata, security, and data quality.

• Lead Terraform infrastructure automation, CI/CD processes, testing, deployment, and environment standards.

• Enhance performance, SLAs, and cloud costs for BigQuery, Dataflow, Spark, storage, and streaming workloads.

• Mentor, guide, and provide technical leadership to Data Engineers, Senior Data Engineers, and Technical Leads.

• Monitor engineering progress, risks, dependencies, and key delivery milestones.

• Serve as the primary technical contact for stakeholders based in the US.

• Work collaboratively with Business, Product, Data Science, BI, DevOps, governance, and security teams.

• Convert business requirements into scalable technical solutions and clearly communicate architecture decisions, risks, timelines, and trade-offs.

⛳️ Requirements

• 15+ years of experience in Data Engineering, Data Architecture, or Cloud Engineering.

• 5+ years of practical GCP Data Engineering experience.

• Strong hands-on experience in AWS Data Engineering and Architecture.

• Demonstrated experience with AWS-to-GCP migration projects.

• Extensive experience in designing enterprise Data Lake/Lakehouse platforms.

• In-depth understanding of AWS and GCP service mapping and cloud migration patterns.

• Proficient in SQL and possess strong skills in Python/PySpark.

• Solid background in data modeling and data warehousing.

• Familiarity with Terraform and CI/CD practices.

• Experience in managing and mentoring data engineering teams.

• Excellent communication skills and experience with US-based stakeholders.

• Capability to evaluate AWS workloads, define GCP target architecture, and lead the migration through to production implementation.

• Expertise in Data Lakehouse, Data Warehouse, Streaming, Data Modeling, and contemporary data engineering patterns.

• Knowledge of data quality, lineage, metadata, IAM, encryption, access control, and enterprise governance.

• Required AWS skills: Amazon S3, AWS Glue, AWS Glue Data Quality, Amazon Redshift/Redshift Serverless, Amazon Athena, AWS Step Functions, AWS DMS, AWS Lake Formation, and IAM.

• Required GCP skills: BigQuery, Google Cloud Storage, Pub/Sub, Dataflow/Apache Beam, Cloud Composer/Airflow, Dataproc/Spark, Cloud Monitoring, Cloud Logging, and Dataplex/Data Catalog.

• Familiarity with Python, SQL, PySpark/Apache Spark, ETL/ELT, CDC, batch and streaming processing, event-driven architecture, and data pipeline optimization.

• Understanding of Enterprise Data Lake/Lakehouse, Medallion Architecture, dimensional and multi-tenant data modeling, schema-on-read/schema-on-write, data lineage, metadata management, data governance, dbt, Apache Airflow, Terraform, Git/GitHub, Cloud Build, CI/CD, and data quality frameworks.

• Knowledge of OpenLineage is an advantage.

🏝️ Benefits

• Flexible remote work environment.

• Exposure to global enterprise customers.

• Collaborative, innovation-driven engineering culture.

• Continuous learning and certification opportunities.