Senior Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Collaborate with extensive big data platforms such as Databricks and Apache Spark.

• Design and uphold ETL/ELT pipelines for the ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.

• Create high-performance data processing jobs utilizing PySpark/Spark and Python on Cloudera and Databricks.

• Enhance pipeline performance and cost efficiency through partitioning, file formats, compute tuning, and effective query patterns.

• Contribute to CI/CD for data workflows, encompassing testing, code reviews, and deployment automation.

• Collaborate with Product Managers to comprehend user needs and use cases.

• Define and develop new modules and features.

⛳️ Requirements

• Extensive hands-on experience in data engineering, constructing production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms, particularly Databricks).

• High-level proficiency in Python, Spark, Hadoop platforms and tools (Hive, Impala, Airflow, NiFi), as well as SQL.

• Practical experience with cloud data platforms such as Databricks and Snowflake; preference for Databricks.

• Familiarity with orchestration/integration tools like Apache Airflow, Apache NiFi, or Talend.

• Working knowledge of DevOps/CI/CD methodologies, including Git, automated testing, release pipelines, and observability.

• Strong problem-solving abilities and aptitude for debugging complex data issues.

• Capacity to communicate effectively with both technical and non-technical stakeholders.

• Resilience, emotional intelligence, and a commitment to agile delivery.

• Profound understanding of the distinction between coding and engineering.

• Advanced proficiency in oral English.

• Advanced proficiency in Spanish.

• Experience in developing Java-based applications is a plus.

• Familiarity with cloud-native foundations or AI coding assistants is preferred.

🏝️ Benefits

• Flexible remote work arrangement.

• Global career opportunities and involvement in high-impact projects.

• Engage in comprehensive digital transformation projects.

• Opportunities for growth through world-class, impactful projects.

• Workplace recognized as one of the most sustainable corporations worldwide.