Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, maintain, and enhance notebooks, pipelines, and data transformation workflows utilizing Databricks.

• Play a key role in the development and advancement of ETL processes, ensuring seamless integration across various data sources.

• Create Python scripts for data processing, validation, and management.

• Construct, optimize, and manage SQL queries and routines associated with SQL Server databases.

• Develop, monitor, and sustain data pipelines, ensuring their availability and reliability.

• Analyze failures, conduct troubleshooting, and implement solutions for processing incidents and issues.

• Contribute to the ongoing enhancement of data quality, governance, and overall performance.

• Apply development best practices, version control, and monitoring for data processes.

• Collaborate with cross-functional teams to deliver data-centric solutions.

⛳️ Requirements

• Practical experience with Databricks in developing, maintaining, and enhancing notebooks and pipelines.

• Strong proficiency in Python for data manipulation, processing, and validation.

• Familiarity with SQL Server, including SQL queries, joins, stored procedures, and query optimization.

• Experience with ETL (Extract, Transform, Load) processes.

• Proven experience in building, maintaining, and monitoring data pipelines.

• Capability to troubleshoot, investigate failures, and resolve issues effectively.

• Knowledge of Azure Data Factory (ADF) for creating, orchestrating, and monitoring pipelines.

• Understanding of the Microsoft Azure ecosystem and its data storage and processing services.

• Familiarity with PySpark, Apache Spark, and Delta Lake.

• Experience integrating various data sources and processing data.

• Proficiency with Git and code version control.

• Awareness of data engineering development best practices.

• Experience in optimizing performance and ensuring data quality.

🏝️ Benefits

• Flexible benefits card – choose how and where to use it.

• Tuition assistance for undergraduate, graduate, MBA, and language courses.

• Certification incentive programs.

• Flexible working hours.

• Competitive salaries.

• Annual performance review with a structured career development plan.

• International career opportunities.

• Wellhub and TotalPass.

• Private pension plan.

• Childcare assistance.

• Medical insurance.

• Dental insurance.

• Life insurance.