Senior Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, develop, and sustain scalable ETL/ELT pipelines for both batch and real-time data processing.

• Construct and enhance data models, Delta Tables, and Lakehouse architectures aimed at analytics and reporting.

• Create and integrate RESTful APIs and data services to facilitate data exchange among enterprise systems.

• Implement frameworks for real-time and high-frequency data ingestion, utilizing streaming technologies and event-driven architectures.

• Design and oversee cloud-native data solutions leveraging Azure Data Factory, Azure Databricks, ADLS, Event Hubs, and Synapse Analytics.

• Develop and fine-tune Databricks Spark applications for extensive data transformation and processing.

• Ensure data quality, governance, security, and compliance across all data platforms.

• Collaborate with data scientists, analysts, application teams, and business stakeholders.

• Troubleshoot, monitor, and enhance pipeline performance and the reliability of data platforms.

• Support DataOps and CI/CD methodologies for the deployment and automation of data pipelines.

⛳️ Requirements

• Strong expertise in SQL and relational databases like Oracle, SQL Server, and MySQL.

• Proficient programming skills in Python, PySpark, PL/SQL, Java, or Scala.

• Practical experience with Azure Cloud technologies, including Azure Data Factory, Azure Databricks, ADLS, Azure Synapse Analytics, Azure Event Hubs, Azure Functions, Azure API Management, and Azure DevOps.

• Familiarity with Databricks Lakehouse architecture, Delta Lake, and Delta Tables.

• Proficient in API development, API integration, RESTful services, and microservices architecture.

• Experience in processing high-volume and high-frequency data with stringent low-latency requirements.

• Strong knowledge of real-time data ingestion and streaming technologies such as Kafka, Azure Event Hubs, or Kinesis.

• Experience with Spark, Hadoop, and distributed data processing frameworks.

• Hands-on experience with OpenShift, Kubernetes, Docker, and containerized deployments.

• Familiarity with workflow orchestration tools like Apache Airflow and Azure Data Factory.

• Understanding of data governance, data security, and compliance best practices.

• Resilience, emotional intelligence, and a commitment to agile delivery.

• Profound understanding of the distinction between coding and engineering.

• Experience with Delta Live Tables (DLT), Auto Loader, and Change Data Capture (CDC) is preferred.

• Knowledge of DataOps, CI/CD, and Infrastructure as Code (IaC) is preferred.

• Familiarity with event-driven architectures and real-time analytics platforms is preferred.

• Azure Data Engineer (DP-203) and Databricks certifications are preferred.

• Familiarity with cloud-native foundations or AI coding assistants is preferred.

• Advanced oral English is required for effective collaboration with global teams.

• Advanced Spanish is required.

🏝️ Benefits

• Flexible work arrangements that support your lifestyle.

• Remote work options based on specific project requirements.

• Global career advancement opportunities.

• Exposure to transformative digital projects with high impact.

• Collaboration with international teams.

• Professional development through participation in world-class, high-impact projects.