Senior Data Engineer

atMellroRemoteBR flagBrazilFreelanceData EngineerMid-levelSenior

Posted 1 day ago

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and enhance the architecture of the platform’s data using Databricks, AWS, and PostgreSQL.

• Create, maintain, and support ETL/ELT pipelines utilizing PySpark, SQL, and AWS services.

• Execute data ingestion processes, including incremental loads and Change Data Capture (CDC).

• Organize and develop the Bronze, Silver, and Gold layers within Delta Lake.

• Model and deliver data that is optimized for utilization in BI tools, with a focus on Power BI.

• Oversee data governance through Unity Catalog, ensuring access control, traceability, and lineage.

• Implement testing protocols and controls for data quality.

• Monitor pipelines, clusters, and databases, identifying failures and opportunities for performance enhancement and cost reduction.

• Address incidents and issues related to data solutions.

• Apply best practices for security, privacy, and compliance with LGPD.

• Organize and maintain CI/CD, code versioning, and infrastructure processes.

• Engage in projects related to the implementation, evolution, and modernization of the data platform.

• Compile technical documentation encompassing architecture, pipelines, integrations, and data models.

• Collaborate with stakeholders to gather requirements and delineate solutions.

• Act as a technical resource for the team by reviewing code, establishing standards, and mentoring less experienced professionals.


⛳️ Requirements

• Minimum of 3 years of experience in Data Architecture or Data Engineering.

• Practical experience in developing and supporting ETL/ELT pipelines in a production environment.

• Strong expertise in Databricks, particularly with Delta Lake, Lakehouse architecture, Unity Catalog, and Workflows/Jobs.

• Familiarity with AWS, especially S3, IAM, RDS/Aurora PostgreSQL, and integration and automation services.

• Advanced understanding of PostgreSQL, including data modeling, indexing, query optimization, and extraction/CDC techniques.

• Strong command of SQL and Python.

• Experience in PySpark and distributed processing.

• Knowledge of data modeling for analytical purposes.

• Experience delivering data to BI tools, preferably Power BI.

• Proficient with Git and collaborative version control practices.

• Understanding of data quality, security, and governance, including LGPD compliance.

• Ability to gather and assess requirements in collaboration with business teams.

• Experience in preparing technical and architectural documentation.

• Databricks and/or AWS certifications are advantageous.

• Familiarity with Terraform and Databricks Asset Bundles is a plus.

• Knowledge of CI/CD and DevOps/DataOps methodologies is a plus.

• Experience in FinOps and cost optimization within Databricks and AWS is a plus.

• Background in data platform migration or modernization initiatives is a plus.

• Experience administering Metabase is a plus.


🏝️ Benefits

• Day off

• Wellness platform

• Cost allowance

• Flexible working hours

People also viewed

Trax Technologies7 hours ago

Data Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Fiserv7 hours ago

Senior Snowflake Data Platform and Artificial Intelligence Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Ontrac Solutions8 hours ago

Staff Data Engineer

US flagUnited States OnlyFreelanceData Engineer
ApplyView job
General Dynamics Information Technology8 hours ago

ICAM Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology8 hours ago

Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology8 hours ago

Senior Data Engineer

US flagUnited States OnlyFull-timeData Engineer$102k – $138k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers