Senior Data Engineer, Data Architecture

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Architect and design solutions in Data Engineering with full ownership of architecture.

• Implement and maintain analytical data platforms in a production environment.

• Address incidents, engage in on-call duties, and operate within high-stakes environments.

• Create modular, testable, and version-controlled code utilizing Python.

• Develop and manage Apache Airflow DAGs, including sensors, backfills, dependencies, and error handling.

• Collaborate with cloud-based MPP Data Warehouses, with a preference for Amazon Redshift.

• Execute versioned and tested transformations using dbt or similar tools.

• Employ Kimball dimensional modeling techniques, covering facts, dimensions, granularity, SCDs, bridge tables, and snapshots.

• Utilize Git, conduct code reviews, and implement CI/CD practices within data projects.

• Analyze and optimize performance, showcasing improvements with metrics before and after changes.

• Preferred qualifications include experience with financial and regulatory market data, AWS/Terraform infrastructure, Lakehouse architectures, streaming/CDC, data quality and cataloging, BI tools, and relevant migration experiences.

⛳️ Requirements

• A minimum of 6 years of experience in Data Engineering, with at least 2 years in an architectural capacity.

• Demonstrated experience in deploying and managing analytical data platforms in production settings.

• Background in handling incidents, on-call responsibilities, and operating in high-criticality environments.

• Advanced SQL skills, encompassing window functions, recursive CTEs, execution plan analysis, and diagnosing data skew and disk spill issues.

• Strong proficiency in Python for Data Engineering, including knowledge of Pandas/Polars, type annotations, and automated testing.

• Hands-on experience with Apache Airflow in production, covering DAGs, sensors, backfills, dependencies, and failure management.

• Familiarity with cloud MPP Data Warehouses, ideally Amazon Redshift; knowledge of Snowflake, BigQuery, or Databricks is advantageous.

• Experience with dbt or similar tools for creating versioned transformations and implementing tests.

• Proficient in Dimensional Modeling (Kimball), including facts, dimensions, granularity, SCD Type 1 and 2, bridge tables, and snapshots.

• Competence in using Git, participating in code reviews, collaborative development, and CI/CD processes in data projects.

• Experience in performance modeling and analysis.

• Nice-to-haves include experience in financial markets; familiarity with regulatory data (CVM, BACEN, BSM, ANBIMA); knowledge of Terraform and AWS; Iceberg, Delta, and Lakehouse; Kafka, Debezium, and Kinesis; Great Expectations, Soda, DataHub, and OpenMetadata; Power BI, Metabase, and Looker; and relevant migration experiences with parity validation.

🏝️ Benefits

• Flash Card - R$ 936.00 monthly for flexible spending on food, meals, and mobility.

• SulAmérica Health Plan - no co-payment or deductions required.

• SulAmérica Dental Plan - no co-payment or deductions required.

• Life Insurance coverage.

• Gympass | Wellhub - Access to gyms and physical activities.

• TotalPass - Access to gyms, fitness activities, wellness programs, and sports.

• SESC partnership - Benefits for cultural, leisure, sports, and wellness activities.

• Creditas - Access to financial benefits, loans, and pension plan options.

• Continuous professional development and exchange of technical talent.

• Opportunities for redeployment to new internal projects.

• Direct interaction with leadership and clients.

• Comprehensive support from the People team and Business Partner (BP).