Lead Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design and construct scalable, cloud-native data platforms and pipelines.

• Develop efficient ETL/ELT pipelines that integrate data from a variety of systems and sources.

• Create production-ready data solutions utilizing Python, Spark, SQL, and Databricks.

• Design modular, maintainable, and scalable components for data processing.

• Engage in architecture and technical design discussions.

• Oversee code reviews while ensuring adherence to engineering best practices, testing, and maintainability.

• Mentor and guide data engineers, fostering technical development and knowledge sharing.

• Support data quality, governance, lineage, cataloging, and access management initiatives.

• Diagnose complex data and pipeline issues within production settings.

• Participate in planning, estimation, feature sizing, and technical discussions with cross-functional teams.

⛳️ Requirements

• Senior or Lead-level experience in Data Engineering.

• Extensive hands-on experience in developing production-grade data engineering solutions.

• Advanced proficiency in Python, PySpark/Spark, and SQL.

• Significant experience with Databricks.

• Practical experience with the Hadoop ecosystem, including Hive and Impala.

• Experience in developing and managing data pipelines using Apache Airflow, Apache NiFi, or Talend.

• Familiarity with ETL/ELT, data integration, and distributed data processing.

• Strong understanding of Git, CI/CD, automated testing, and release pipeline processes.

• Experience in troubleshooting and optimizing complex data pipelines and workloads.

• Experience in a technical lead role or as a senior technical contributor within data engineering teams.

• Proven experience in mentoring engineers and conducting technical/code reviews.

• Ability to break down complex data problems into scalable and maintainable solutions.

• Involvement in architecture and platform design decisions.

• Strong communication skills to interact with both technical and non-technical stakeholders.

• Proficiency in conversational English and Spanish is essential; candidates will be evaluated.

• For foreign applicants residing in Mexico, valid INE, CURP, NSS, and RFC are required.

• Must be based in Mexico.

• Must possess a visa to work in Mexico.

🏝️ Benefits

• Excellent superior benefits.