Senior Data Engineer, Python, Snowflake, Airflow, AWS

This is a fully remote position, open to applicants in Portugal.

📋 Description

• Design and manage data-centric systems that handle extensive datasets on AWS.

• Prioritize Snowflake optimization, including data modeling, performance enhancement, and cost-effectiveness.

• Create and orchestrate intricate workflows using Apache Airflow.

• Develop data transformations with dbt, encompassing modeling, testing, and documentation.

• Implement both real-time and batch data pipelines utilizing AWS services such as S3.

• Oversee core cloud infrastructure via Infrastructure as Code with Terraform.

• Ensure engineering excellence through CI/CD pipelines utilizing GitHub Actions.

• Execute thorough automated testing (unit, linting, integration, end-to-end).

• Employ observability tools like New Relic.

• Collaborate effectively using Git/GitHub workflows.

⛳️ Requirements

• Proficient in Python development with the ability to guide and mentor fellow engineers.

• Advanced SQL skills with experience in managing large-scale datasets.

• Extensive knowledge of Snowflake, including data modeling, optimization, and performance tuning.

• Strong grasp of architectural principles, microservices design, and data-centric systems.

• Practical experience with Apache Airflow for designing, orchestrating, and monitoring workflows.

• Familiarity with dbt for data modeling, transformations, and testing.

• Proven track record in building and operating both real-time and batch data pipelines.

• Direct experience with AWS S3 and essential cloud services.

• Solid experience with GitHub Actions or similar CI/CD technologies.

• Skilled in using Git/GitHub and collaborative development workflows.

• Experience utilizing Terraform for infrastructure management.

• Strong practices in unit testing, linting, integration, and end-to-end testing.

• Familiarity with structured, semi-structured, and unstructured data.

• Experience with monitoring platforms like New Relic or comparable technologies.

• Fluent in English (mandatory).

🏝️ Benefits

• Flexible work arrangements.

• Opportunities for professional development.