Principal Data Engineering Consultant

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Provide innovative analytical solutions for clients globally

• Assist in constructing robust, dependable data platforms that support AI initiatives

• Create comprehensive data engineering solutions across various simultaneous projects

• Collect requirements and offer consulting services to clients

• Implement Microsoft Azure and cloud computing principles

• Develop solutions utilizing Apache Spark, Databricks, Microsoft Fabric, or alternative Spark engines

• Primarily code solutions in Python

• Write intricate, high-performance SQL queries and comprehend query execution across different computing technologies

• Utilize Azure Data Factory and other cloud orchestration tools

• Work with Azure Data Lake storage and Delta Lake

• Employ data governance tools such as Unity Catalog and Purview

• Use infrastructure-as-code tools like Terraform and Azure DevOps

• Oversee agile projects, including budget and scope management

• Implement ETL practices, data quality measures, resilience, observability, CI/CD, and SDLC methodologies

• Collaborate within a diverse team and contribute to knowledge sharing within the broader data community

• Participate in occasional visits to clients and team gatherings

⛳️ Requirements

• Must be located within the UK

• Strong comprehension of Microsoft Azure and cloud computing principles

• Experience with Apache Spark, including Databricks, Microsoft Fabric, or other Spark engines

• Proficient in Python

• Skilled in SQL, including complex, high-performance queries and understanding query execution across various computing technologies

• Familiarity with Azure Data Factory or other cloud orchestration tools

• Experience with Azure Data Lake storage and Delta Lake

• Knowledge of Unity Catalog, Purview, or other data governance tools

• Familiarity with an IaC tool such as Terraform or Azure DevOps

• Capable of gathering requirements and providing client consulting

• Experience in agile project management, including managing budgets and scope

• Understanding of object-oriented development practices

• Knowledge of ETL methodologies, including data quality, resilience, and observability

• Familiarity with CI/CD practices and the software development life cycle (SDLC)

• Ability to visualize the end-to-end solution across multiple concurrent projects

• A technical passion coupled with integrity and empathy in client consulting

• Willingness to share expertise, skills, and insights internally and with the broader data community

🏝️ Benefits

• Industry-standard Maternity & Paternity policy

• Private Medical & Dental Care along with an Eye care scheme

• Health & Wellbeing allowance

• 25 Days Annual Leave, along with Birthday Leave & Charity Leave

• Team away days

• Discretionary Annual Bonus Scheme

• Flexible Working Hours

• Electric Car Scheme

• Access to the latest technology and training opportunities

• Commitment to employees’ long-term goals