Senior Data Engineer – Platform Modernization, Migration

This is a fully remote position, open to applicants in Argentina, +4 more countries.

📋 Description

• Oversee the complete migration of around 1,000 datasets to Snowflake.

• Upgrade orchestration workflows from Apache Airflow 2 to Airflow 3.

• Transition legacy Hive data structures and datasets to Apache Iceberg utilizing the Snowpark Connect framework/connectors.

• Reconfigure, update, and transfer Confluent/Apache Kafka streaming connections across various cloud regions.

• Modify existing data pipelines to align with Snowflake environments.

• Enhance SQL queries through the use of CTEs, window functions, and intricate joins.

• Oversee, test, and troubleshoot Apache Airflow DAGs and manage workflow scheduling.

• Develop testing strategies and perform hands-on code debugging to guarantee data quality and consistency.

• Modernize streaming infrastructure across cloud regions.

• Maintain rigorous data quality and performance standards throughout platform modernization.

⛳️ Requirements

• At least 2+ years of practical data engineering experience with large-scale datasets, Apache Airflow, and Apache Spark.

• Solid foundation in software engineering principles.

• Practical experience in Python development.

• Advanced SQL capabilities, including complex joins, CTEs, window functions, and query optimization.

• Demonstrated experience with Snowflake data modeling and schema design.

• Direct experience in constructing and testing pipelines using Apache Airflow, Apache Kafka, and Apache Spark/PySpark.

• Proficiency in Apache Iceberg and Apache Parquet data formats.

• Familiarity with AWS S3, including data lakes, bucket partitioning, and lifecycle management policies.

• Ability to establish testing strategies and independently troubleshoot complex distributed data pipelines.

• Excellent communication skills and capability to work autonomously in a dynamic environment.

• Client-provisioned AWS WorkSpace is required.

• Must demonstrate good connectivity to an AWS WorkSpace region by completing the connectivity test and providing a screenshot as evidence.

• Proficiency in English is required for all interviews, documentation, and daily interactions.

🏝️ Benefits

• Fully remote work arrangement.

• Full-time, permanent employment.

• Client-provisioned AWS WorkSpace.