AWS Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, develop, and maintain automated ETL pipelines utilizing Python and Apache Airflow.

• Orchestrate pipelines to execute either sequentially or in parallel.

• Monitor pipeline operations and reprocess records that encounter errors.

• Diagnose data pipeline errors and proactively address issues.

• Import and export CSV, JSON, and XML data primarily from S3 or other cloud storage solutions.

• Enhance complex SQL queries for improved performance in data warehouse technologies, particularly Snowflake.

• Generate code through Python configurations and Jinja templates.

• Establish CI/CD processes from the ground up using GitHub.

• Collaborate with international teams across various global delivery centers.

• Contribute to AI-driven automation and cloud solutions for clients in finance, healthcare, and manufacturing.

⛳️ Requirements

• Over 10 years of total experience with a fully hands-on approach to the necessary technical stack.

• Advanced expertise in Python concepts, including code structures, modules, packages, classes, subclasses, inheritance, multi-threading, and functional programming.

• Experience in developing reusable Python packages.

• Strong SQL skills, including complex joins, CTEs, and subqueries.

• Proficiency in Python unit, integration, and system testing.

• Competency in implementing DBT tests for data validation and quality assurance.

• Extensive knowledge of AWS S3, ECS, and IAM.

• Familiarity with AWS security best practices, IAM roles, encryption standards, secure coding guidelines, and DBT profile access configurations.

• Experience with AI IDE tools such as Cursor or GitHub Copilot.

• Ability to develop automated ETL processes and schedule jobs such as Airflow DAGs.

• Proven experience troubleshooting and resolving data pipeline errors.

• Competence in exporting/importing CSV, JSON, and XML data from S3 or other cloud storage solutions.

• Experience optimizing SQL queries specifically for Snowflake.

• Experience in code generation using Python and Jinja templates.

• Familiarity with GitHub and establishing CI/CD processes from scratch.

• Advanced oral English skills for effective collaboration with global teams.

• Proficient in Spanish.

• A high-performance mindset, resilience, emotional intelligence, and a focus on agile delivery.

• Desired: Experience with AWS Lambda.

• Desired: Experience with Apache Airflow on AWS, including integration of AWS services and DBT models.

• Desired: API integrations with Salesforce, SFTP, SharePoint, OneDrive, and similar platforms.

• Desired: Development of complex DBT models and custom DBT macros using Jinja and Python.

• Desired: Familiarity with AWS CloudWatch and Datadog.

• Desired: Understanding of cloud-native foundations or experience with AI coding assistants.

🏝️ Benefits

• Flexible work arrangement to support your lifestyle.

• Option for remote work.

• Opportunities for continuous learning.

• Engagement with high-impact projects and complex engineering challenges.

• Opportunities for growth and professional development.

• Collaboration with international teams and global delivery centers.