Data Platform Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and sustain the GCP data platform by establishing standards for data ingestion, processing, and orchestration.

• Develop and enhance batch and near-real-time ingestion pipelines sourced from transactional databases, APIs, and events.

• Seamlessly integrate new data sources into BigQuery using a standardized approach that minimizes implementation effort over time.

• Create and enforce data contracts along with schema governance.

• Implement comprehensive end-to-end data observability, focusing on lineage, freshness, volume, and alerting mechanisms.

• Manage version control of infrastructure and resources using Terraform, maintain CI/CD pipelines, and expedite the process from concept to data availability in production.

• Enhance BigQuery performance and manage costs effectively at scale by utilizing partitioning, clustering, reservations, and consumption monitoring.

• Safeguard data security and privacy through effective access controls, data masking, and retention policies.

• Serve as a technical advisor and mentor for the team, guiding architecture choices, conducting code reviews, and promoting best practices.

• Collaborate closely with Analytics Engineering and Data Science to provide the necessary foundation and tools for autonomy.

⛳️ Requirements

• Possess technical seniority to make informed architecture decisions, weighing explicit trade-offs, and influence teams beyond your immediate group.

• Demonstrate strong SQL skills, with the ability to compose and troubleshoot complex, efficient, and readable queries.

• Have substantial experience with Python in a data context: constructing pipelines, integrating APIs and internal tools, and producing tested, production-grade code.

• Acquire practical experience with BigQuery, Cloud Storage, Pub/Sub, and processing services such as Dataflow, Dataproc, or similar technologies.

• Familiarity with Airflow/Cloud Composer, including concepts of idempotency, backfills, and dependency management.

• Proficient in Terraform, Git/GitHub, CI/CD, and conducting code reviews.

• Have a solid understanding of data warehousing and dimensional modeling principles.

• Experience with Dataform or dbt and the lifecycle of versioned transformations.

• Background in automated data testing, monitoring, and incident response.

• Preferred: familiarity with streaming technologies like Pub/Sub, Dataflow, Kafka, or Flink in low-latency environments.

• Preferred: knowledge of data mesh, data catalogs, and the establishment of self-service capabilities for business teams.

• Preferred: understanding of data architecture for RAG.

• Preferred: experience with Kubernetes, containers, and SRE practices applied to data pipelines.

• Preferred: experience within the financial services or fintech industry, including knowledge of compliance and auditing.

🏝️ Benefits

• National health insurance with no monthly premium or co-payments for our #Simplers and up to two eligible dependents.

• National dental insurance with no monthly premium or co-payments for our #Simplers and up to two eligible dependents.

• Flexible food allowance provided through the iFood Benefícios card.

• Home-office allowance available on the iFood Benefícios card.

• Wellhub benefit extended to eligible dependents (children and spouses).

• A day off for your birthday.

• Partnerships with educational institutions for continued learning.

• Zenklub (Mental Health) – access to a mental health platform offering four free therapy sessions each month for every Simpler.

• Life insurance coverage.

• Extended parental leave, including 180 days of maternity leave and 45 days of paternity leave, which also applies to adoptive parents.

• Childcare allowance for children up to one year old.

• Onhappy (Travel) – travel benefits offering exclusive discounts on hotels, flights, and experiences.

• Guapeco (Pet Health) – partnership for pet healthcare with special conditions.