Data Platform Engineer, GCP

This is a fully remote position, open to applicants in United States.

📋 Description

• Assist in the management and upkeep of production GCP data platforms, pipelines, and workflows that encompass both batch and streaming workloads.

• Provide support for Cloud Composer / Apache Airflow workflows, addressing issues related to DAG failures, dependencies, scheduling, retries, latency, and SLA compliance.

• Enhance the reliability, execution speed, recoverability, and operational efficiency of Airflow DAGs and orchestration workflows.

• Diagnose and resolve pipeline failures, data latency issues, performance declines, configuration challenges, and infrastructure problems.

• Oversee platform health and work on improving reliability, scalability, performance, and operational efficiency.

• Refine pipelines and workloads to boost refresh times, throughput, query performance, and overall platform stability.

• Assist with platform, infrastructure, configuration, and dependency upgrades while ensuring production stability and compliance.

• Conduct root-cause analyses and implement lasting solutions for recurring production challenges.

• Offer support for IAM, access controls, monitoring, alerting, logging, and operational governance.

• Work collaboratively with clients and cross-functional teams in Data Engineering, BI/Analytics, Application Engineering, Infrastructure, and Platform domains.

• Adjust to client-specific engineering, security, compliance, review, and operational processes.

• Evaluate architectures and suggest incremental enhancements.

• Produce and maintain technical documentation, operational runbooks, troubleshooting guides, and platform support procedures.

⛳️ Requirements

• 3–7+ years of experience in Data Engineering, Cloud Engineering, Platform Engineering, or a similar position.

• Extensive hands-on experience in building, supporting, or managing production workloads on Google Cloud Platform (GCP).

• Strong proficiency with BigQuery and Cloud Spanner.

• Practical experience with Dataflow, Pub/Sub, batch/streaming data pipelines, and Cloud Composer / Apache Airflow.

• Solid skills in Python and SQL.

• Familiarity with GCP IAM, service accounts, permissions, monitoring, logging, alerting, and production operations.

• Experience in troubleshooting intricate production environments and conducting root-cause analyses.

• Knowledge of data ingestion, transformation, orchestration, data quality, performance optimization, and reliability.

• Competence in understanding existing systems, codebases, pipelines, configurations, and client-specific tools and workflows.

• Excellent communication and collaboration abilities across engineering and business teams.

• Preferred: experience with large-scale business-critical platforms, App Engine, Cloud Run, GKE, CI/CD, DevOps, Terraform, data observability, automated data-quality monitoring, Vertex AI, and enterprise analytics tools.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible working hours and the option for remote work.

• Opportunities for professional development and continuous learning.

• A collaborative and inclusive company culture.