Senior GCP Data Engineer

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Design and oversee batch and streaming data pipelines within the Google Cloud Platform (GCP) environment;

• Create ingestion and transformation workflows utilizing Dataflow and Apache Beam;

• Consolidate data from various sources, including Oracle, SQL Server, MySQL, on-premises setups, and external buckets;

• Utilize BigQuery for data processing, transformation, and optimization tasks;

• Structure and organize data in Cloud Storage adhering to a Data Lake architecture (Raw, Bronze, Silver, and Gold);

• Manage workflows and pipelines through Cloud Composer (Airflow);

• Develop ETL/ELT processes and analytical transformations using SQL;

• Create automations and data integrations with Python;

• Ensure the integrity and reliability of processed data;

• Assist in migrating reports and analytical assets to a new Data Visualization platform;

• Collaborate with Engineering, Analytics, Data Science teams, and business stakeholders.

⛳️ Requirements

• Proven experience with Google Cloud Platform (GCP);

• Familiarity with BigQuery;

• Hands-on experience with Dataflow and Apache Beam;

• Proficiency in Cloud Storage and Pub/Sub;

• Experience with Cloud Composer (Airflow) or similar orchestration tools;

• Python expertise for developing pipelines and ETL/ELT processes;

• Advanced SQL skills for data transformation and manipulation;

• Experience in integrating data from relational databases such as Oracle, SQL Server, and MySQL;

• Understanding of Data Lake architecture (Raw, Bronze, Silver, and Gold);

• Experience in building and maintaining data pipelines;

• Knowledge of Dataform, dbt, or comparable tools (preferred);

• Familiarity with streaming data processing (preferred);

• Understanding of data quality and governance best practices (preferred);

• Experience in BI, Analytics, or Data Visualization migration projects (preferred);

• Knowledge of performance and cost optimization in GCP environments (preferred).

🏝️ Benefits

• Health insurance

• Retirement plan

• Flexible working arrangements

• Opportunities for professional development