This is a fully remote position, open to applicants in United Kingdom.

• Develop and sustain ETL pipelines and code utilizing cloud data platforms.

• Assist with the ingestion processes and onboarding of new data sources.

• Collaborate with the Databricks Data Intelligence Platform to create Data Engineering workloads.

• Build raw, refined, and curated data layers, ensuring proper asset cataloging.

• Validate solutions against both functional and non-functional specifications.

• Provide datasets, transformations, and data products that are optimized for performance.

• Enhance processes, engineering methodologies, and reusable tools.

• Monitor and evaluate pipeline performance; aid in resolving incidents.

• Ensure that documentation adheres to acceptance standards and is centrally approved.

• Participate actively in Agile ceremonies and governance discussions.

• Extensive experience in Python, PySpark, and SQL for data engineering tasks.

• Practical experience with Azure Databricks.

• In-depth knowledge of Azure Data Factory and Fabric.

• Familiarity with Microsoft Purview.

• Knowledge of event-driven data ingestion mechanisms (Event Grid / Pub-Sub).

• Understanding of SOLID principles, asynchronous programming, and Mediator/Factory design patterns.

• Experience in delivering unit and integration testing within Databricks.

• Knowledge of secure ETL design, including Entra IMID/SCIM integration.

• Awareness of Azure best practices, APIM, and platform governance.

• Capability to build and serve Power BI models using Databricks data sources.

• Prior experience in UK Public Sector environments.

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and training.

• Flexible working arrangements and a supportive work environment.

• Comprehensive healthcare and wellness programs.

Data Engineer

People also viewed