Mid/Senior Data Engineer, Consulting

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Serve as a consultant on strategic business initiatives centered around data engineering utilizing Databricks.

• Recognize and implement best practices for high-volume data ingestion, ensuring optimal performance and availability.

• Enhance queries and data modeling to boost the efficiency of analytical queries.

• Create and maintain ETL/ELT data pipelines to assimilate various sources, ensuring data quality and governance.

• Design and execute Data Lakehouse architectures adhering to the Medallion pattern (Bronze, Silver, Gold).

• Assist with data modeling for contemporary Data Warehouses (Star Schema, Snowflake).

• Implement best practices in Big Data settings, particularly with Databricks and Delta Lake.

• Utilize Delta Lake, Iceberg, Parquet, and Avro within Lakehouse frameworks.

• Ensure proper data governance, including access controls and versioning.

⛳️ Requirements

• Extensive experience with Databricks and Delta Lakehouse architecture.

• Proficient SQL skills and a record of optimizing queries for distributed processing.

• Background in ETL/ELT and in designing scalable data architectures.

• Familiarity with incremental data pipelines (UPSERTS, MERGE, CDC).

• Expertise in analytical data modeling (Star Schema, Snowflake).

• Experience with Databricks ingestions, including pipeline refactoring and performance optimization.

• Knowledge of Unity Catalog for governance and access control within Databricks.

• Understanding of modern Data Warehouses (Databricks, Snowflake, BigQuery, Redshift).

• Practical experience with AWS architectures, including S3, IAM, Glue, and EMR.

• Experience in provisioning and configuring Databricks clusters from scratch, covering infrastructure, networking, permissions, and integration with data catalogs.

🏝️ Benefits

• Flexibility: remote-first team, enabling work from any location.

• Health and dental insurance with Bradesco.

• Access to Conexa Saúde & Psicologia Viva.

• Wellhub (formerly Gympass) membership.

• Partnership with Open English for language learning.

• Caju: Home office allowance provided.

• Birthday day off to celebrate your special day.

• 22 business days each year of paid company recess.

• Contracting model: PJ (fixed monthly fee).