Cloud Engineer – Azure, Databricks

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Take charge of the administration, configuration, maintenance, and optimization of Azure Databricks environments, which includes workspaces, clusters, and runtimes.

• Establish and implement best practices for Databricks administration, platform architecture, security, reliability, and cost management.

• Act as the main infrastructure partner for Data Engineering teams, ensuring platform availability and operational stability.

• Proactively monitor cloud infrastructure and address performance, availability, and configuration challenges.

• Respond to high-severity incidents, emergency support requests, and Databricks cluster failures.

• Design, implement, and sustain infrastructure automation utilizing Terraform, Bicep, and CI/CD pipelines.

• Configure and uphold secure Azure network architectures, including VNet Injection, Private Endpoints, and NSGs.

• Implement and manage platform security, access controls, and governance through Unity Catalog and Microsoft Entra ID.

• Troubleshoot intricate distributed computing and cloud platform issues, identify root causes, and develop sustainable solutions.

• Collaborate with Data Engineering and Infrastructure teams to meet platform requirements and ensure reliable data processing environments.

• Independently assess infrastructure requirements, make technical decisions, and foster continuous enhancements in platform operations.

• Optimize the utilization of cloud resources, enhance infrastructure performance, and improve operational efficiency.

⛳️ Requirements

• Must be currently residing in Mexico.

• At least 3 years of experience in Cloud Engineering, Platform Engineering, or Infrastructure Engineering, with a strong emphasis on Microsoft Azure.

• Significant hands-on experience in administering Azure Databricks, including workspace provisioning, cluster management, runtime configuration, and platform lifecycle management.

• Experience in defining and executing Databricks architecture, administration, security, and cost-optimization best practices.

• Strong expertise in Infrastructure as Code (IaC) tools such as Terraform or Azure Bicep.

• Proficient in Python, Bash, or PowerShell.

• Comprehensive understanding of Azure networking and security, including VNet Injection, Private Endpoints, Network Security Groups (NSGs), and secure Databricks deployment methodologies.

• Familiarity with cloud infrastructure automation and CI/CD tools like Azure DevOps or GitHub Actions.

• Knowledge of access management and governance through Unity Catalog and Microsoft Entra ID (Azure AD).

• Demonstrated ability to troubleshoot complex cloud infrastructure and distributed computing issues.

🏝️ Benefits

• Stock options.

• Options for remote work.

• Flexible working hours.

• Benefits that exceed legal requirements.

• Collaboration with seasoned IT professionals.

• Opportunities for mentorship and personal development.