Senior Data Engineer – Azure Databricks Platform

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Assume technical leadership for essential workstreams on EnerMech's Azure Databricks data platform, focusing on data modeling, pipeline development, Azure infrastructure, CI/CD, DevOps, and DataOps.

• Design, construct, ensure quality, and manage assigned workstreams from initial scoping to production deployment.

• Uphold the integrity of bronze/silver/gold medallion data models for designated data domains.

• Model, validate, document, and certify data for downstream reporting and analytics.

• Develop and optimize Delta tables and materialized views for scalable performance.

• Create and implement data quality frameworks, including quarantine/error handling and validation against data contracts.

• Define compute configurations tailored to workloads, balancing performance with cost considerations.

• Contribute to and troubleshoot Terraform-managed infrastructure, including networking, storage, Key Vaults, Managed DevOps Pools, and Dev Centers.

• Oversee and maintain Azure DevOps build and release pipelines, variable groups/libraries, repository security, permissions, and branch policies.

• Assist with environment-level Azure configuration, subscription access, RBAC, service principal identities, and Terraform state backend management.

• Address complex platform issues and document architectural decisions.

• Leverage AI-assisted tools for development, validating outputs against authoritative sources.

• Serve as the technical point of contact for the global EBS Team and business stakeholders.

• Engage with technical and non-technical stakeholders across the UK, Americas, Middle East, and Asia Pacific time zones.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related discipline, or equivalent practical experience.

• Typically 6+ years in data engineering, with at least 3+ years of hands-on experience with Azure Databricks in a production environment.

• Proven ownership of medallion architecture design and data modeling.

• Experience implementing data quality frameworks and data contracts in a production setting.

• Familiarity with PII protection methods (masking, tokenization, or access tiering) across environments.

• Experience in platform engineering: diagnosing infrastructure issues rather than merely consuming platform services.

• Demonstrated ability to deliver autonomously, including direct engagement with stakeholders.

• Databricks Certified Data Engineer Associate (minimum requirement).

• Databricks Certified Data Engineer Professional (desirable).

• Microsoft AZ-400: Designing and Implementing Microsoft DevOps Solutions (desirable).

• Knowledge of multi-environment Azure networking (VNet peering, DNS zone design, NAT Gateway) (desirable).

• Experience in migrating or establishing CI/CD pipelines, including the introduction of pipeline templates/standards (desirable).

• Understanding of governance and identity patterns (e.g., Entra ID, RBAC scoping) (desirable).

• Ability to apply regulatory frameworks (e.g., GDPR) to data platform design (desirable).

• Experience in a shared service center or global delivery model, preferably within energy or industrial services (desirable).

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and certifications.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• A collaborative and inclusive work environment.