This is a fully remote position, open to applicants in Brazil.

• Develop, enhance, and expand data pipelines and infrastructure utilizing Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake.

• Create, operationalize, and oversee ingest and transformation workflows, including DAGs, alerting, retries, SLAs, lineage, and cost management.

• Partner with platform and AI/ML teams to automate ingestion, validation, and real-time compute workflows, progressing towards a feature store.

• Incorporate pipeline health and metrics into engineering dashboards for comprehensive visibility and observability.

• Design data models and execute efficient, scalable transformations in Snowflake and PostgreSQL.

• Develop reusable frameworks and connectors to standardize the internal data publishing and consumption processes.

• Over 4 years of hands-on data engineering experience in production settings.

• Extensive practical experience with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines.

• Proficient SQL skills and experience managing PostgreSQL in live environments.

• Strong understanding of cloud-native data workflows (preferably AWS) and pipeline observability (metrics, logging, tracing, alerting).

• Demonstrated experience managing pipelines from start to finish: design, implementation, testing, deployment, monitoring, and iteration.

• Experience in tuning Snowflake performance and optimizing costs.

• Familiarity with real-time or near-real-time processing (e.g., streaming ingestion, incremental models, CDC).

• Practical experience with a backend TypeScript framework (NestJS is a significant advantage).

• Knowledge in data quality frameworks, contract testing, or schema management.

• Experience in building internal developer platforms or components of data platforms (connectors, SDKs, CI/CD for data).

• Fully remote position.

• Work hours aligned with EST or PT time zones.

Data Engineer, Python, PySpark, AWS Glue, Amazon Athena, SQL, Apache Airflow

People also viewed