Senior Data Engineer, GCP, Data Integration

This is a fully remote position, open to applicants in Peru.

📋 Description

• Design, maintain, and improve production ETL pipelines that connect REST APIs with GCP-based data warehouse environments.

• Create and sustain data integration components utilizing TypeScript/Node.js and Python within the existing pipeline architecture.

• Construct data pipelines using BigQuery, Cloud Storage, Dataflow, and Pub/Sub.

• Implement transformation logic for billing events, subscription lifecycle data, revenue data, and other data entering the GCP data warehouse.

• Develop and sustain REST API integrations, including authentication, pagination, error handling, retries, and dependable data extraction.

• Enhance pipeline monitoring and reliability through alerting, logging, error management, retry mechanisms, and data quality assessments.

• Write intricate SQL queries for validating source-to-warehouse data, reconciliation, and parity checks.

• Collaborate with architecture stakeholders on warehouse-level validation requirements and convert them into SQL checks and reconciliation logic.

• Investigate and resolve issues related to data pipelines, integration, transformation, and data quality.

• Adhere to established coding standards, architectural patterns, CI/CD workflows, and development practices within the existing codebase.

• Work alongside Data Engineering leadership for technical guidance while retaining ownership of assigned development tasks.

• Deliver clear written status updates, identify blockers promptly, and communicate progress effectively within an async-first distributed team.

• Engage in Agile/Kanban workflows while maintaining a focus on hands-on engineering execution.

⛳️ Requirements

• A Bachelor’s Degree in Computer Science, Software Engineering, Data Engineering, Information Systems, or a related field is preferred, or equivalent professional experience.

• Over 6 years of professional experience in Data Engineering, ETL, Data Integration, or a closely related domain.

• Extensive hands-on experience with TypeScript/Node.js in production ETL or data integration settings.

• Strong Python proficiency in developing production data pipelines using Dataflow, Airflow, or Cloud Composer.

• Hands-on experience with GCP data services, including BigQuery, Cloud Storage, Dataflow, and Pub/Sub.

• Proven ability to build and maintain REST API integrations, including OAuth 2.0 authentication, pagination, error handling, and retry logic.

• Advanced SQL competencies, including complex queries across warehouse schemas for data validation, reconciliation, and analysis.

• Familiarity with data transformation logic and production data pipelines.

• Capability to comprehend and navigate within an existing architecture, adhering to established coding patterns, conventions, and CI/CD workflows.

• Strong troubleshooting and problem-solving abilities across data pipelines, integrations, and cloud environments.

• Excellent written communication skills and the capacity to provide clear updates in an async-first, distributed setting.

• A self-directed approach to onboarding and the ability to become effective within an existing codebase with minimal supervision.

• Working proficiency in English for collaboration with US-based stakeholders and distributed teams.

• Professional experience with the Zuora REST API is a plus.

• Familiarity with ZOQL or Zuora’s billing data model, including subscriptions, invoices, and payments, is a plus.

• Experience with dbt or similar data transformation frameworks is a plus.

• GCP Professional Data Engineer certification is a plus.

🏝️ Benefits

• Option to work remotely.

• Chance to engage in AI-powered solutions and scalable products.

• Collaborative opportunities across design, development, cloud, data, and artificial intelligence.

• Opportunities for learning and ownership.

• Distributed, async-first working environment.

• Agile/Kanban workflows.

• Professional development through engagement with modern technologies and AI-native transformation.