Data Engineer, LATAM – Python, PySpark, AWS Glue, Amazon Athena, SQL, Apache Airflow

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop, enhance, and scale data pipelines and infrastructure utilizing Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake.

• Create, operationalize, and oversee data ingestion and transformation workflows: DAGs, alerting, retries, SLAs, lineage, and cost management.

• Work collaboratively with platform and AI/ML teams to automate ingestion, validation, and real-time computing workflows; aim towards establishing a feature store.

• Incorporate pipeline health and metrics into engineering dashboards to ensure complete visibility and observability.

• Structure data and execute efficient, scalable transformations in Snowflake and PostgreSQL.

• Develop reusable frameworks and connectors to standardize the internal publishing and consumption of data.

⛳️ Requirements

• Over 4 years of experience in production data engineering.

• Extensive, hands-on expertise with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines.

• Strong SQL capabilities and experience managing PostgreSQL in live environments.

• Comprehensive understanding of cloud-native data workflows (preferably AWS) and pipeline observability (metrics, logging, tracing, alerting).

• Demonstrated experience managing pipelines from start to finish: design, implementation, testing, deployment, monitoring, and iteration.

🏝️ Benefits

• Compensation will be provided in USD.

• Work hours are set according to the EST time zone (9 AM to 6 PM EST) or PT time zone.