Mid-Level Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Serve as a Mid-Level Data Engineer at Zé Labs, leading Data Engineering and Analytics Engineering projects within the team.

• Enhance the team’s independence regarding pipelines, data ingestion, quality assurance, and observability.

• Guarantee that data for analytical products is accessible, dependable, documented, and sustainable.

• Collaborate with Analytics Engineers to construct and sustain analytical models, including the Silver layer and semantic layer.

• Develop, maintain, and improve data pipelines.

• Integrate data from various sources such as relational databases, events, messaging systems, and files.

• Organize and process data within the Bronze layer.

• Empower Zé Labs to have greater control over pipelines and processes reliant on the corporate Data team.

• Define, review, and validate the infrastructure for jobs, workflows, and pipelines.

• Evaluate code and technical solutions, advocating for engineering best practices.

• Mentor and assist junior team members.

• Implement and enhance logs, metrics, alerts, and monitoring for pipeline health.

• Investigate issues related to failures, inconsistencies, and performance.

• Establish error handling, retries, idempotency, reprocessing, and backfills.

• Document sources, pipelines, models, business rules, dependencies, and data contracts.

• Contribute to SLAs, refresh criteria, and data quality benchmarks.

• Assess downstream impacts prior to modifying schemas, pipelines, or models.

• Engage in prioritizing and managing the technical backlog alongside the Data Manager/Data PM.

• Take part in technical discussions with Data Engineering, Platform Engineering, and stakeholders.

⛳️ Requirements

• Completed Bachelor’s degree.

• Degrees in Technology, Computer Science, Engineering, Statistics, Mathematics, or related fields are preferred but not mandatory.

• Ideally three or more years of experience in Data Engineering, Analytics Engineering, or related disciplines.

• Open to remote work.

• Proficient in Python for developing pipelines, integrations, or automations.

• Skilled in SQL for data querying, transformation, and modeling.

• Practical understanding of Spark/PySpark or similar distributed processing technologies.

• Experience in building and maintaining data pipelines.

• Familiarity with integrating sources such as relational databases, files, APIs, events, or messaging systems.

• Experience with contemporary data platforms or lakehouse architectures.

• Knowledge of Git, pull requests, and code review methodologies.

• Experience with error handling, retries, reprocessing, and backfills.

• Practical knowledge of testing, observability, and monitoring of pipelines.

• Familiarity with Databricks, Delta Lake, and Unity Catalog is considered an asset.

• Experience with AWS and cloud data services is advantageous.

• Knowledge of Databricks Workflows, Airflow, or similar orchestration tools is a plus.

• Understanding of medallion architecture, CI/CD for data pipelines, data contracts, schema evolution, impact analysis, job optimization, partitioning, compaction, processing costs, and security practices is beneficial.

🏝️ Benefits

• Medical Insurance

• Dental Insurance

• Telemedicine

• Life Insurance

• Gympass

• Discount on company products

• Christmas basket

• Toys for employees’ children

• Private pension plan

• Meal or Food Voucher

• Optional transportation allowance

• Attendance bonus (14th salary)

• Childcare or Babysitter Allowance

• Profit Sharing

• Meritocratic and inclusive environment

• Opportunities for development and career growth