Senior Data Engineer – Google Cloud

Posted 4 days ago

This is a fully remote position, open to applicants in United States, +2 more locations.

πŸ“‹ Description

β€’ Evaluate the existing data estate of the customer and develop a target-state GCP data architecture along with a migration strategy.

β€’ Create ingestion, transformation, and orchestration pipelines, which include both batch and streaming pipelines, into BigQuery.

β€’ Implement data infrastructure entirely using Terraform and CI/CD methodologies.

β€’ Transition historical data and workloads, ensuring documented reconciliation to demonstrate row- and aggregate-level alignment.

β€’ Optimize BigQuery models and queries for cost efficiency and performance improvement.

β€’ Equip the customer with tools for monitoring and alerting on the health of the pipelines.

β€’ Provide runbooks and architectural documentation for future reference.

β€’ Conduct knowledge transfer sessions to empower the customer team to manage the platform post hand-off.

β€’ Collaborate closely with customer engineering and analytics teams from the discovery phase through to the hand-off.


⛳️ Requirements

β€’ Over 6 years of experience in data engineering, with at least 3 years focused on delivering production workloads on Google Cloud Platform.

β€’ Extensive hands-on experience with BigQuery, including data modeling, partitioning and clustering, cost and performance optimization, and ELT using SQL.

β€’ Proven experience in building and managing batch and streaming pipelines utilizing Dataflow (Apache Beam) and/or Dataproc (Spark / PySpark), with Pub/Sub as a streaming source.

β€’ Proficient in orchestrating production workflows with Cloud Composer (Apache Airflow), encompassing dependency management, retries, and alerting mechanisms.

β€’ Strong skills in Python and advanced SQL.

β€’ Experience in provisioning GCP data infrastructure using Terraform and deploying via Git-based CI/CD.

β€’ Successfully completed at least one migration into BigQuery from an on-premises or legacy platform, including data validation and cut-over processes.

β€’ Client-facing experience, including conducting requirements sessions, presenting designs, and handing over documented solutions to customer teams.

β€’ Possession of Google Cloud Professional Data Engineer certification or current Associate Cloud Engineer / Professional Cloud Architect certification (preferred qualification).

β€’ Databricks certification or relevant production experience with Databricks on GCP (preferred qualification).

β€’ Experience with Looker / LookML modeling, or delivering dashboards using Looker Studio (preferred qualification).

β€’ Involvement in serverless integration projects on Cloud Run or Cloud Functions (preferred qualification).

β€’ Knowledge of data governance on GCP, including Dataplex, Data Catalog, IAM and column-level security, and DLP (preferred qualification).

β€’ Experience in enabling GenAI use cases from the data warehouse: BigQuery ML, Vertex AI, embeddings, or RAG data preparation (preferred qualification).

β€’ Hands-on experience with BigQuery Migration Service / SQL translation, Datastream, or Database Migration Service during an actual migration (preferred qualification).

β€’ Previous work experience with a Google Cloud partner or within Google Cloud Professional Services (preferred qualification).

β€’ No sponsorship is available.


🏝️ Benefits

β€’ Fully remote engagement available from anywhere in the United States or Canada.

β€’ Potential for contract extension beyond the initial 6-month period.

People also viewed

Trax Technologies6 hours ago

Data Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Fiserv6 hours ago

Senior Snowflake Data Platform and Artificial Intelligence Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Ontrac Solutions6 hours ago

Staff Data Engineer

US flagUnited States OnlyFreelanceData Engineer
ApplyView job
General Dynamics Information Technology6 hours ago

ICAM Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology6 hours ago

Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology6 hours ago

Senior Data Engineer

US flagUnited States OnlyFull-timeData Engineer$102k – $138k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers