Senior Data Engineer – Airflow, AWS

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, sustain, and enhance data pipelines leveraging Apache Airflow (MWAA);

• Create processes for data ingestion, transformation, and provisioning (ETL/ELT);

• Develop solutions utilizing Python, Spark, and AWS services;

• Implement and maintain data architectures within Data Lake, Lakehouse, and Data Warehouse settings;

• Conduct data modeling to fulfill analytical and business requirements;

• Guarantee data quality, governance, and observability of processed information;

• Oversee and enhance the performance of data pipelines and workflows;

• Collaborate with business, analytics, and technology teams to advance the data platform;

• Assist in defining best practices, standards, and ongoing improvements throughout the data ecosystem;

⛳️ Requirements

• Extensive experience with Apache Airflow (MWAA) for orchestrating data pipelines;

• Proficient in Python development for creating and maintaining ETL/ELT processes;

• In-depth knowledge of SQL Server;

• Experience with Apache Spark for large-scale data processing;

• Familiarity with AWS services, especially Redshift, S3, Glue, and MWAA;

• Understanding of data modeling for analytical environments;

• Experience with Data Warehouse, Data Lake, and Lakehouse architectures;

• Knowledge of Medallion architecture (Bronze, Silver, Gold);

• Experience in data governance, quality, and observability;

• Background in developing, optimizing, and monitoring data pipelines;

🏝️ Benefits

• Meal allowance or meal voucher;

• Discounts on courses, universities, and language schools;

• Stefanini Academy — a platform offering free, current online courses and certificates;

• Mentoring;

• Benefits club for consultations and exams;

• Health insurance;

• Dental insurance;

• Discount and perks club at premier retailers and establishments;

• Travel club;

• Pet care benefits.