Senior Data Engineer – AWS, Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Analyze data from various source systems, pinpointing structures, relationships, and business rules.

• Oversee the transfer of raw data into the Databricks environment, ensuring data integrity, traceability, and quality.

• Create and sustain scalable, efficient data pipelines (ETL/ELT) on Databricks and Delta Lake while exercising technical autonomy.

• Implement business rules and transformations necessary for generating regulatory reports.

• Construct and verify reports for BACEN (Brazilian Central Bank), ensuring compliance with layouts, validations, and regulatory timelines.

• Participate in data architecture decisions and the design of solutions within the Databricks/AWS landscape.

• Guarantee adherence to compliance, security, governance, and data quality standards throughout the data lifecycle.

• Troubleshoot and resolve intricate technical challenges associated with the Databricks platform, offering advanced support.

• Provide technical assistance and share expertise with less experienced team members.

• Work collaboratively with business, regulatory, and cross-functional technical teams.

⛳️ Requirements

• Minimum of 4 years of experience as a data engineer, showcasing proven senior-level proficiency.

• Extensive experience with Databricks (ideally 3+ years), Python, and AWS.

• Skilled in integrating Databricks within an AWS cloud environment.

• Strong understanding of data modeling, Delta Lake, data warehousing, and ETL/ELT methodologies.

• Proficient in Python and SQL for processing substantial data volumes.

• Experience in designing and implementing distributed data architectures and processing pipelines.

• Practical experience in mapping and migrating data from legacy systems or various sources.

• Ability to independently drive end-to-end deliveries and make informed technical choices.

• Focused on quality, scalability, performance, and data governance.

• Effective communication and collaboration skills within cross-functional teams.

• Capable of creatively and efficiently solving problems.

• Nice to have / Differentials:

• Familiarity with regulatory reporting to BACEN (e.g., SCR / Documento 3040, COSIF, DLO) and/or background in the financial/banking sector.

• Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional) or certifications in related technologies.

• Knowledge of DevOps practices and infrastructure automation (CI/CD).

• Experience with data visualization tools (Power BI, MicroStrategy).

• Exposure to agile methodologies (Scrum/Kanban).

• Experience mentoring junior/mid-level professionals.

🏝️ Benefits

• 🏥 Porto Seguro Health Plan

• 🦷 Porto Seguro Dental Plan

• 💰 Profit Sharing (PLR)

• 👶 Childcare Assistance

• 🍽️ Alelo Meal and Food Vouchers

• 💻 Home Office Allowance

• 📚 Partnerships with Educational Institutions

• 🚀 Support for Certifications, including Cloud

• 🎁 Livelo Points

• 🏋️‍♂️ TotalPass

• 🧘‍♂️ Mindself