Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, construct, and oversee robust, scalable ETL/ELT pipelines utilizing Python, Spark, and Apache Airflow for substantial volumes of financial data.

• Architect, implement, and enhance the data environment on Google Cloud Platform using Terraform.

• Manage, monitor, and scale hybrid MongoDB and MySQL databases.

• Ensure 99.99% availability, disaster recovery, and smooth data flow for on-premise MySQL instances that support Mexico's SPEI payment system.

• Detect data bottlenecks, enhance complex queries, optimize indexes, and manage partitioning for millisecond-level latencies.

• Incorporate Generative AI tools into daily operations to expedite data modeling, automate ETL scripting, and analyze infrastructure logs.

⛳️ Requirements

• Over 4 years of experience as a Data Engineer, Data Platform Engineer, or in a comparable position, with a solid understanding of Database Administration (DBA) in high-transaction environments.

• Extensive hands-on experience with GCP, including Cloud Storage, Compute Engine, Cloud SQL/Spanner, and more.

• Proven experience orchestrating intricate workflows using Apache Airflow.

• Advanced capability in Python for data processing tasks.

• Proficiency in Bash scripting.

• Experience with Terraform/Ansible for automated deployments.

• Strong practical experience in administering and tuning MySQL, MongoDB, and distributed databases like SingleStoreDB or equivalent.

• Expertise in execution plans, query optimization, high availability, and database replication strategies, including cross-site replication and Group Replication.

• Experience managing critical financial systems, large-scale transactional data, or payment gateways.

• Native or fully fluent in Spanish, with conversational and technical proficiency.

• At least 4 years of demonstrable experience in building data pipelines, automating processes, and overseeing data infrastructure.

• Solid and verifiable experience with Google Cloud Platform and Apache Airflow.

• Advanced proficiency in Python and Infrastructure as Code tools, primarily Terraform.

• Practical experience managing MySQL, MongoDB, and distributed databases.

• Proven track record of ensuring 99.99% uptime, executing Disaster Recovery Plans (DRP), and overseeing database replication strategies.

• Regular daily use of Generative AI tools to develop data scripts, optimize queries, or automate repetitive tasks.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• Comprehensive health, dental, and vision insurance plans.

• A dynamic and inclusive work environment that fosters innovation.