Data Engineer – Airflow, AWS

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, maintain, and improve data pipelines utilizing Apache Airflow (MWAA)

• Create data ingestion, transformation, and delivery workflows (ETL/ELT)

• Build solutions leveraging Python, Spark, and AWS services

• Implement and support data architectures within Data Lake, Lakehouse, and Data Warehouse settings

• Conduct data modeling to fulfill analytical and business requirements

• Ensure the quality, governance, and observability of processed data

• Monitor and enhance the performance of data pipelines and processes

• Collaborate with business, analytics, and technology teams to advance the data platform

• Aid in defining best practices, standards, and ongoing improvements throughout the data ecosystem

⛳️ Requirements

• Extensive experience with Apache Airflow (MWAA) for orchestrating data pipelines

• Proficient in Python development for creating and maintaining ETL/ELT workflows

• Advanced proficiency in SQL Server

• Familiarity with Apache Spark for handling large datasets

• Practical experience with AWS services, particularly Redshift, S3, Glue, and MWAA

• Understanding of data modeling for analytical settings

• Experience with Data Warehouses, Data Lakes, and Lakehouses

• Knowledge of Medallion Architecture (Bronze, Silver, and Gold)

• Background in data governance, quality, and observability

• Experience in developing, optimizing, and monitoring data pipelines

🏝️ Benefits

• Meal or food allowance

• Discounts on courses, universities, and language institutions

• Stefanini Academy — a platform providing free, up-to-date online courses with certificates

• Mentoring opportunities

• Benefits club offering discounts on consultations and medical tests

• Health insurance

• Dental insurance

• Benefits and discounts at leading establishments