Senior DataOps Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Oversee and provide support for Databricks workspaces, cluster policies, compute configurations, jobs, workflows, libraries, service principals, permissions, and platform integrations.

• Facilitate Unity Catalog access governance, including catalog and schema permissions, as well as secure data sharing.

• Monitor and resolve issues related to Delta Live Tables, Spark jobs, Delta Lake workloads, and data pipelines.

• Ensure secure and dependable integrations between Databricks and AWS services, such as S3, IAM, networking, VPC connectivity, and external data sources.

• Assist with the AWS infrastructure utilized by the data platform, encompassing EC2, VPC, IAM, security groups, networking, patching, and operational access.

• Provide support for MariaDB/MySQL and MongoDB environments, focusing on availability, monitoring, connectivity, backup coordination, performance troubleshooting, and recovery.

• Develop and maintain monitoring systems, dashboards, logs, metrics, and alerts across platforms, pipelines, databases, infrastructure, and integrations.

• Engage in on-call support and lead investigations into production incidents, including mitigation, recovery, communication, and root-cause analysis.

• Construct and sustain infrastructure and automation utilizing Terraform, Terraform Enterprise, Git, GitHub Actions, CI/CD pipelines, and scripting.

• Automate tasks such as provisioning, access configuration, monitoring deployment, recovery, and platform changes.

• Administer secure access controls, secrets, certificates, TLS configuration, service accounts, and IAM roles.

• Enhance operational readiness through the creation of runbooks, recovery procedures, architecture diagrams, standards, escalation paths, and technical documentation.

• Collaborate with engineering and infrastructure teams on capacity planning, platform performance, cost visibility, and resource optimization.

• Investigate practical applications of AI and automation to enhance troubleshooting, documentation, incident response, and operational efficiency.

⛳️ Requirements

• Significant experience in supporting production data platforms, cloud infrastructure, or DataOps environments.

• Practical experience in administering and supporting Databricks, including workspaces, jobs, workflows, compute, cluster policies, and access controls.

• Familiarity with supporting Spark, Delta Lake, and Delta Live Tables workloads.

• Extensive experience with AWS services such as EC2, VPC, IAM, S3, security groups, CloudWatch, and networking.

• Proficient in infrastructure as code using Terraform and Terraform Enterprise or similar tools.

• Experience in supporting MariaDB/MySQL and MongoDB within production settings.

• Strong grasp of monitoring, observability, alerting, and incident management practices.

• Familiarity with CI/CD, Git, GitHub Actions, and automation using Bash, Python, PowerShell, or other scripting languages.

• Competence in troubleshooting platform performance, connectivity, permissions, certificates, TLS, and integration issues.

• Background in production support, root-cause analysis, post-incident reviews, and on-call operations.

• Comprehensive understanding of security and access concepts such as IAM, RBAC, secrets management, certificates, and secure connectivity.

• Excellent communication and collaboration skills, capable of working across Data Engineering, Analytics, Security, Infrastructure, Database, and application teams.

• Experience with Unity Catalog, Databricks SQL, Databricks CLI, REST APIs, and Databricks asset or deployment bundles.

• Knowledge of Kubernetes administration and production troubleshooting.

• Experience with monitoring tools like Prometheus, Grafana, Datadog, or similar platforms.

• Familiarity with Rootly or similar incident management platforms.

• Experience with MongoDB Atlas, Ops Manager, or comparable MongoDB tools.

• Proficient in MariaDB/MySQL replication, schema migrations, indexing, backup validation, and restore procedures.

• Experience in capacity planning, cloud cost optimization, and workload performance tuning.

• Background in implementing or enhancing data governance, compliance controls, or audit processes.

• Experience utilizing AI-assisted tools to enhance operations, troubleshooting, documentation, or automation.

🏝️ Benefits

• Medical Insurance

• Savings Fund

• Life Insurance

• Internet and Electricity Allowance

• Paid time off, including vacation and parental leave

• Competitive compensation and benefits package designed to support physical, financial, and professional well-being