Senior Data Engineer

This is a fully remote position, open to applicants in Philippines.

📋 Description

• Develop, construct, and manage data ingestion pipelines for financial and billing data utilizing GCP.

• Create and enhance BigQuery data models, incorporating nested ARRAY/STRUCT tables, as well as strategies for partitioning and clustering.

• Execute reconciliation checks across source, staging, and output layers.

• Establish file validation and quarantine protocols for invalid or incomplete source data.

• Design idempotent pipelines that can be safely executed multiple times without causing duplication or corruption.

• Implement append-only table patterns and utilize views for data verification to ensure auditability.

• Accurately manage monetary values using BigQuery NUMERIC types and apply appropriate rounding rules.

• Oversee effective-dated reference data such as rates, mappings, and customer attributes.

• Create and sustain data lineage from outputs back to sources.

• Write clean, typed, and thoroughly tested Python code employing pydantic and pytest.

• Develop and optimize advanced SQL for transformation, reconciliation, and reporting purposes.

• Collaborate with engineers, analysts, and finance stakeholders to convert business rules into dependable data logic.

• Mentor colleagues and contribute to establishing data engineering standards and best practices.

⛳️ Requirements

• Over 6 years of experience in data engineering.

• Proven track record in building financial or billing pipelines that require every amount to reconcile.

• Strong expertise in Python, including familiarity with pydantic, typed code, and pytest.

• Advanced SQL capabilities, encompassing complex joins, window functions, and performance optimization.

• Comprehensive experience with Google BigQuery, including nested ARRAY/STRUCT tables, partitioning, and clustering techniques.

• Proficient in accurately managing money using NUMERIC data types.

• Experience in designing append-only tables and leveraging views for data validation.

• A solid understanding of data quality practices, including reconciliation checks, file validation, and quarantine processes.

• Experience in developing idempotent, rerunnable pipelines.

• Expertise in implementing data lineage and managing effective-dated reference data.

• Familiarity with the invoicing domain, including credit notes, rebills, FX conversion, and month-end close processes.

• Background working with finance or billing teams in a regulated or audited setting.

• Experience with additional GCP services such as Cloud Storage, Cloud Run, or Firestore.

🏝️ Benefits

• Competitive compensation package reflective of experience.

• Government-mandated benefits.

• Supplemental HMO coverage.

• Collaborative and professional work environment.

• Opportunities for career advancement within a growing organization.

• Hybrid/Remote work options (where applicable).

• Training programs to support employee development and career progression.