Senior Data Engineer, AWS

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and maintain ETL/ELT data pipelines in high-volume, complex settings utilizing Apache Spark (Scala and Python) and AWS;

• Play a key role in the development and enhancement of the Analytics Platform, emphasizing Big Data architectures and technologies;

• Collect, analyze, and translate business requirements into effective technical data solutions;

• Create and uphold analytical data models, metrics, and KPIs;

• Integrate various data sources, including streams (Apache Kafka / MSK) as well as SQL and NoSQL databases;

• Develop, operate, and advance data pipelines within production environments;

• Ensure scalability, performance, reliability, data quality, and governance;

• Prepare, cleanse, and make data accessible for analytical and operational use;

• Automate the creation, configuration, and monitoring of data pipelines and their environments;

• Oversee SLAs, manage failure remediation, and resolve incidents within data flows;

• Collaborate within agile teams, supporting technical decisions and advocating for data engineering best practices;

• Offer technical mentorship and share insights with team members.

⛳️ Requirements

• Extensive experience with Amazon Web Services, especially serverless architectures, Data Lake, and event-driven integration;

• Practical experience with storage, orchestration, messaging, streaming, data cataloging, and NoSQL databases in the AWS ecosystem;

• Proficient in building and maintaining both batch and streaming pipelines, with a focus on orchestration and failure handling;

• Strong proficiency in Python (primary development language);

• Experience with PySpark for data processing;

• Advanced SQL skills (daily use in Athena);

• Experience with MySQL (modeling, performance, and troubleshooting);

• Organizing Data Lake into a structured layout: raw / trusted / refined;

• Familiarity with Parquet and JSON formats, partitioning, and read optimization;

• Experience in data integration via RESTful APIs;

• Proven track record in critical environments, emphasizing SLAs, reliability, and observability;

• Ownership mindset, proactive approach, and active participation in architectural decisions.

🏝️ Benefits

• Bullet point 1

• Bullet point 2