Senior Data Engineer – AWS, Data Platforms

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Participate in an international nearshore project, aiding in the development, support, and modernization of data platforms within the financial services industry.

• Design, create, maintain, and enhance scalable and dependable data pipelines.

• Develop data ingestion, integration, and transformation processes utilizing SQL, Python, and distributed processing technologies.

• Create and maintain AWS-based data solutions leveraging Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, and AWS Lambda.

• Assist in the development and evolution of Data Lakes, Data Warehouses, and contemporary data platforms.

• Implement ETL and ELT processes that are aligned with both technical and business requirements.

• Develop solutions using Apache Spark or PySpark to handle large datasets.

• Engage in architecture discussions and contribute to the creation of secure, scalable, and sustainable solutions.

• Monitor and enhance the performance of pipelines, queries, transformations, and cloud workloads.

• Investigate and address issues related to data ingestion, processing, storage, and delivery.

• Conduct root cause analyses and implement corrective measures.

• Ensure the quality, integrity, security, and governance of data.

• Collaborate with architects, software engineers, analysts, and business stakeholders.

• Support cloud migration initiatives, platform modernization, and the evolution of data architecture.

• Implement automation, version control, CI/CD, and DataOps practices.

• Ensure adherence to technical standards, internal controls, and regulatory requirements.

• Create and maintain technical documentation, data flows, and operational procedures.

• Contribute to standardization, knowledge sharing, and continuous improvement.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Software Engineering, or a related area.

• Advanced proficiency in English (C2), capable of communicating in an international context.

• Proven experience in designing, developing, and supporting scalable data pipelines.

• Advanced skills in SQL and Python.

• Familiarity with ETL and ELT processes, as well as data ingestion, integration, and transformation.

• Experience with AWS data services, including Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, and AWS Lambda.

• Understanding of data modeling and architectures for Data Warehouses and Data Lakes.

• Experience with distributed processing using Apache Spark or PySpark.

• Ability to develop, monitor, optimize, and troubleshoot data pipelines.

• Knowledge of data quality, security, and governance practices.

• Experience working in Agile settings and collaborating with multidisciplinary and distributed teams.

• An analytical mindset, independence, clear communication skills, and a strong focus on quality delivery.

• Preferred: experience with Databricks or Snowflake; Apache Airflow or similar tools; dbt; Kafka or other messaging and streaming technologies; Terraform and Infrastructure as Code; Git, automation, and CI/CD; DataOps practices; as well as data platform migration, modernization, or transformation projects.

• Preferred: experience in financial institutions or other regulated environments.

• Relevant certifications in AWS, Azure, Databricks, Snowflake, or data engineering.

🏝️ Benefits

• Medical and dental insurance.

• Childcare support.

• Access to a benefits platform and exclusive EY discounts.

• Educational incentives.

• A day off for your birthday.

• Gympass/Wellhub access.

• Profit-sharing opportunities.

• Private pension plan.

• Meal and/or food and transportation vouchers.

• Flexible work arrangements.

• Programs and groups focused on diversity and inclusion.

• Opportunities for future-focused skills development and world-class experiences.