Azure Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, develop, and maintain scalable ETL/ELT pipelines for both batch and real-time data processing.

• Construct and enhance data models, Delta Tables, and Lakehouse architectures for analytics and reporting purposes.

• Create and integrate RESTful APIs and data services across various enterprise systems.

• Implement real-time and high-frequency data ingestion utilizing streaming technologies and event-driven architectures.

• Design and oversee cloud-native data solutions leveraging Azure Data Factory, Azure Databricks, ADLS, Event Hubs, and Synapse Analytics.

• Develop and refine Databricks Spark applications for extensive data transformation and processing.

• Ensure data quality, governance, security, and compliance are maintained across all data platforms.

• Collaborate effectively with data scientists, analysts, application teams, and business stakeholders.

• Troubleshoot, monitor, and enhance pipeline performance and the reliability of the data platform.

• Support DataOps and CI/CD practices to facilitate the deployment and automation of data pipelines.

⛳️ Requirements

• Strong proficiency in SQL and relational databases including Oracle, SQL Server, and MySQL.

• Solid programming skills in Python, PySpark, PL/SQL, Java, or Scala.

• Practical experience with Azure Cloud technologies such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure Synapse Analytics, Azure Event Hubs, Azure Functions, Azure API Management, and Azure DevOps.

• Familiarity with Databricks Lakehouse architecture, Delta Lake, and Delta Tables.

• Expertise in API development, integration, RESTful services, and microservices architecture.

• Experience in processing high-volume and high-frequency data with low-latency requirements.

• Strong knowledge of real-time data ingestion and streaming technologies including Kafka, Azure Event Hubs, or Kinesis.

• Experience with Spark, Hadoop, and distributed data processing frameworks.

• Hands-on experience with OpenShift, Kubernetes, Docker, and containerized deployments.

• Familiarity with workflow orchestration tools like Apache Airflow and Azure Data Factory.

• Understanding of data governance, security, and compliance best practices.

• Azure Data Engineer (DP-203) and Databricks certifications are mandatory.

• Advanced proficiency in English.

• Mandatory skills include Python and Azure SQL.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional growth and development.

• Access to cutting-edge technologies and tools.

• Collaborative and inclusive work environment.

• Flexible work hours and remote work options.