Senior Data Engineer, Python, AWS, ML Pipelines

This is a fully remote position, open to applicants in Portugal.

📋 Description

• Developing and sustaining data and machine learning pipelines within a production environment

• Creating ETL and data processing workflows utilizing Python

• Managing workflows with tools like Airflow and AWS Step Functions

• Utilizing AWS Glue for data processing and execution of pipelines

• Assisting in the deployment of machine learning models into production through SageMaker

• Collaborating closely with Data Scientists and ML Engineers to operationalize ML models

• Monitoring, diagnosing, and enhancing pipeline reliability and performance

• Participating in architectural and technical decisions regarding data platforms and ML infrastructure

• Functioning in an agile setting with distributed teams

⛳️ Requirements

• Extensive experience with Python

• Proven experience in building data and/or ML pipelines in a production setting

• Practical experience with Airflow

• Familiarity with AWS Step Functions and AWS Glue

• Experience in deploying ML models using AWS SageMaker

• Strong proficiency with AWS cloud services

• Experience managing production systems at scale

• Experience collaborating within cross-functional teams alongside Data Scientists

• Excellent communication skills and capability to work within a distributed team environment

• Proficient in English communication, both written and verbal

🏝️ Benefits

• A culture that focuses on learning from mistakes rather than criticizing failure

• An Agile environment that encourages your ideas

• Continuous Training & Mentoring

• Opportunities for travel