Lead Data Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Take ownership of the technical strategy for data engagements.

• Establish benchmarks for pipeline architecture, data modeling, testing, and operational readiness.

• Lead and guide a team of data engineers, managing work distribution, reviewing designs and code, and resolving technical obstacles.

• Design and enhance the data platform architecture on Microsoft Fabric, which includes data pipelines, Azure OneLake, and the broader Azure ecosystem.

• Implement and uphold CI/CD, testing, and operational practices utilizing Azure DevOps.

• Foster professional relationships with key client stakeholders and convert business and data requirements into a technical roadmap.

• Diagnose and resolve complex issues within live data pipelines.

• Oversee incident response during issues that arise.

• Collaborate with CreateFuture’s broader engineering capabilities to ensure delivery that is both technically excellent and commercially viable.

⛳️ Requirements

• Extensive hands-on experience with modern cloud data platforms such as Microsoft Fabric, Databricks, Snowflake, or Synapse, including the design of production data platforms.

• Profound SQL expertise along with a solid understanding of data modeling and architecture, encompassing Kimball, data vault, medallion/lakehouse patterns, or domain modeling.

• Strong skills in Python and PySpark for large-scale batch/ELT data processing.

• Experience in leading or technically guiding a team of data engineers.

• Background in establishing standards for pipeline design, testing, and delivery.

• Proven experience in managing CI/CD and operational practices for data engineering workflows, preferably using Azure DevOps.

• Direct experience working with client stakeholders.

• Familiarity with the broader Azure ecosystem, including Azure OneLake and Azure ML, especially for model industrialization.

🏝️ Benefits

• Flexible working arrangements, including hybrid and remote options.

• Supportive and friendly workplace culture.

• Commitment to professional and personal development.

• Occasional travel to client locations or CreateFuture offices as necessary.

• Transparent, fair, and respectful hiring process.

• Application adjustments or support are available.

• Recognized as a Great Place to Work-Certified™ company.

• Awarded Best Workplaces UK recognition.