Senior Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Create, develop, and maintain data ingestion pipelines for financial and billing information on Google Cloud Platform (GCP)

• Design and enhance BigQuery data models, incorporating nested ARRAY/STRUCT tables, along with partitioning and clustering strategies

• Execute reconciliation checks across source, staging, and output layers

• Develop file validation and quarantine procedures for any invalid or incomplete source data

• Design idempotent pipelines that can be safely re-executed without risking data duplication or corruption

• Apply append-only table patterns and utilize views as data verification methods to ensure auditability

• Manage monetary values accurately by employing BigQuery NUMERIC types and relevant rounding rules

• Oversee effective-dated reference data such as rates, mappings, and customer attributes

• Establish and maintain data lineage tracing outputs back to their original sources

• Write clean, typed, and thoroughly tested Python code utilizing pydantic and pytest

• Compose and optimize advanced SQL for transformation, reconciliation, and reporting purposes

• Collaborate with engineers, analysts, and finance stakeholders to convert business rules into reliable data logic

• Guide team members and contribute to the development of data engineering standards and best practices

⛳️ Requirements

• Over 6 years of experience in data engineering

• Demonstrated experience in constructing financial or billing pipelines where every amount must reconcile

• Proficient in Python, including pydantic, typed code, and pytest

• Advanced SQL proficiency, encompassing complex joins, window functions, and performance optimization

• Extensive experience with Google BigQuery, including nested ARRAY/STRUCT tables, partitioning, and clustering techniques

• Proven ability to handle monetary values accurately using NUMERIC data types

• Experience in designing append-only tables and utilizing views for data verification

• Strong grasp of data quality practices, including reconciliation checks, file validation, and quarantine procedures

• Background in developing idempotent, rerunnable pipelines

• Experience in implementing data lineage and managing effective-dated reference data

• Preferred experience in invoicing domains, such as credit notes, rebills, FX conversion, and month-end close

• Preferred experience collaborating with finance or billing teams within regulated or audited environments

• Preferred familiarity with other GCP services like Cloud Storage, Cloud Run, or Firestore

🏝️ Benefits

• Opportunities for training and employee development

• Potential for career growth and advancement

• Employees are valued and respected

• A positive and supportive company culture