Senior Data Engineer – Databricks, DBT

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and develop the dbt platform architecture on Databricks, incorporating Unity Catalog and the medallion layers: bronze, silver, and gold.

• Organize the dbt project by establishing modeling standards, naming conventions, layers (staging, intermediate, marts), macros, tests, and documentation.

• Create CI/CD pipelines for dbt that include validation, building, testing, and deployment for various environments such as development, staging, and production.

• Streamline provisioning and operations through Infrastructure as Code (IaC) using Terraform and DevOps methodologies.

• Enhance cost efficiency and performance by managing clusters, SQL Warehouses, Photon, partitioning, and incremental processing.

• Establish data governance and security measures, including Unity Catalog, access control, and masking of sensitive information.

• Guide the team in analytical engineering best practices while reviewing code through pull requests.

⛳️ Requirements

• Extensive experience with dbt Core, encompassing incremental models, snapshots, macros (Jinja), tests, packages, and exposures (minimum of 4 years).

• Practical experience with Databricks, specifically Unity Catalog, SQL Warehouses, clusters, Delta Lake, and Workflows (minimum of 4 years).

• Proficient in SQL and dimensional modeling techniques (Kimball, star schema); familiarity with Data Vault is an advantage.

• Knowledge of CI/CD practices for data using GitHub Actions, GitLab CI, or Azure DevOps.

• Proficient in Git and collaborative workflows, including trunk-based or GitFlow, pull requests, and code reviews.

• Experience with DevOps and IaC, particularly Terraform, environment management, secrets, and configuration.

• Scripting skills in Python.

• Commitment to a culture of automated testing, version control, and reproducible deployments.

🏝️ Benefits

• 🏥 Porto Seguro Health Plan

• 🦷 Porto Seguro Dental Plan

• 💰 Profit Sharing (PLR)

• 👶 Childcare Allowance

• 🍽️ Alelo Meal and Food Vouchers

• 💻 Home Office Allowance

• 📚 Partnerships with Educational Institutions

• 🚀 Support for Certifications, including Cloud

• 🎁 Livelo Points

• 🏋️‍♂️ TotalPass

• 🧘‍♂️ Mindself