Data Platform Engineer

This is a fully remote position, open to applicants in Colombia, +4 more countries.

📋 Description

• Lead and implement the migration of data platform workloads, pipelines, and storage from legacy corporate infrastructure to enterprise cloud infrastructure.

• Transition workloads and infrastructure to AWS us-east-2, ensuring cross-network connectivity, security prerequisites, and adherence to deployment standards.

• Modernize Hadoop, Hive, and Airflow batch ecosystems towards Snowflake, Confluent Kafka, and AWS-managed services.

• Establish governed and secure data access models for downstream systems and business consumers.

• Operate, maintain, and monitor both production and staging environments across Snowflake, Confluent Kafka, and AWS.

• Utilize Terraform and GitLab CI/CD to codify, audit, and automate configurations, platform changes, and environment setups.

• Engage in production troubleshooting, on-call incident response, and root cause analyses.

• Write operational runbooks and migration documentation.

• Collaborate with distributed engineering teams, ensuring at least six hours of overlap during US Central Time business hours.

⛳️ Requirements

• Over 3 years of hands-on, senior-level experience in production data platform engineering with a strong focus on DevOps / Platform Engineering.

• Strong programming skills in Python, shell scripting (Bash), and advanced SQL.

• Production operational/DevOps experience with both Snowflake and Confluent/Kafka.

• In-depth expertise in either Snowflake and AWS with solid operational knowledge of Kafka, or Confluent/Kafka and AWS with strong operational knowledge of Snowflake.

• Experience in operating, deploying, and migrating production-grade data pipelines across development, staging, and production tiers.

• Proficiency in least-privilege security architectures, enterprise change control, secure secrets management, and audited deployment practices.

• Strong skills in debugging, handling production incidents, authoring runbooks, and communicating with cross-functional stakeholders.

• Ability and necessity to perform all development work within an enterprise-provisioned AWS WorkSpaces virtual desktop environment.

• Reliable network latency/connectivity to an AWS WorkSpace region; must complete the official health check and provide a passing-results screenshot.

• All interviews, technical documentation, daily team syncs, and operational communications must be conducted in English.

• Nice-to-have: experience with Apache Spark, Hadoop, Hive/HQL, Airflow, Snowpark, AWS Glue, data lakehouse formats, data observability platforms, corporate integrations, acquisitions, or system alignments.

🏝️ Benefits

• 100% remote contractor position.

• Direct contract with In All Media.

• Minimum 6-hour overlap with US Central Time business hours.