Senior Analytics Engineer, Databricks Lakehouse

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Identify and catalog legacy Data Warehouse consumers (Business Intelligence, reports, applications, and system integrations), categorizing them by domain and importance;

• Design consumption marts within the gold layer, based on the silver layer, adhering to the medallion framework and data agreements;

• Implement the transition of dashboards, reports, applications, and system integrations to the gold layer in phases, aligned with domain cutover schedules;

• Ensure consumption consistency between legacy and gold environments and assist in the phased retirement of legacy consumption points;

• Assist in the prioritization of implementation waves alongside the Product Owner and Data Architect.

⛳️ Requirements

• Analytics Engineering: substantial experience in the consumption layer — including dimensional modeling/marts, semantic layers, and certified metrics;

• Databricks: proficient in production Spark SQL, Delta Lake, and medallion architecture (bronze, silver, gold);

• BI repointing (Power BI, Tableau, or similar): modifying data sources, refactoring datasets, and ensuring numeric consistency;

• System data integrations: experience with APIs, batch extractions, and system loads, including interface contracts and rollback strategies;

• Advanced SQL: skilled in reconciliation techniques and large-scale dataset comparisons;

• Unity Catalog (tracking, discovery, and documentation of metrics and consumption models);

• Dimensional modeling (Kimball/star schema) and adherence to best practices for domain marts;

• Proficient in Python and PySpark for automating parity validation and repointing processes;

• Understanding of Data Contracts and Data Quality (expectations, DQ gates) relevant to the consumption layer;

• Experience in Data Warehouse migration, including legacy/new coexistence, wave-based cutovers, and high-volume dataset reconciliation — a key differentiator;

• Background in Data Mesh initiatives (domains, data as a product, federated governance) — a significant differentiator;

• Familiarity with DataStage or other legacy ETL tools (beneficial for understanding current consumption);

• Experience with semantic layers and metric catalogs (metric stores, semantic layers);

• Knowledge of data observability and quality monitoring within the consumption layer;

• Experience in the financial or credit industry;

• Databricks certifications (Data Analyst Associate, Data Engineer Associate);

• Proficiency in open data contract standards (Open Data Contract Standard, ODPS);

🏝️ Benefits

• Competitive salary and performance-based bonuses;

• Comprehensive health insurance plans;

• Opportunities for professional development and training;

• Flexible work schedule and remote work options;

• Collaborative and inclusive work environment;