Senior GCP Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain scalable batch and real-time data pipelines on GCP.

• Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing.

• Build high-performance data transformations utilizing Python and PySpark.

• Develop and optimize intricate SQL queries for analytical workloads.

• Collaborate extensively with BigQuery for large-scale data processing and performance enhancement.

• Create and deploy pipelines using Cloud Dataflow.

• Orchestrate workflows with Cloud Composer (Apache Airflow).

• Manage data storage and its lifecycle through Google Cloud Storage (GCS).

• Implement version control and CI/CD pipelines using Git-based tools.

• Ensure data security, governance, and access control using GCP IAM.

• Optimize data solutions for performance, scalability, reliability, and cost-efficiency.

⛳️ Requirements

• Strong hands-on experience with Google Cloud Platform (GCP).

• Expertise in BigQuery (including partitioning, clustering, and query optimization).

• Proven experience in implementing Medallion Data Architecture.

• Strong programming abilities in Python and PySpark.

• Practical exposure to Java.

• Advanced proficiency in SQL (covering complex joins, window functions, and performance tuning).

• Hands-on experience with Cloud Dataflow.

• Familiarity with Cloud Composer (Airflow) for orchestration.

• Experience in working with Google Cloud Storage (GCS).

• Knowledge of version control systems (Git) and CI/CD methodologies.

• Strong understanding of GCP IAM and best practices for cloud security.

🏝️ Benefits

• Health insurance.

• Flexible work arrangements.