Senior Data Engineer – Databricks

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Create and manage scalable batch and near real-time data pipelines within Databricks, integrating data from legacy systems, POS, web, app, CRM, and various other sources.

• Construct unified customer profiles, facilitating identity resolution, deduplication, enrichment, and the creation of Golden Records.

• Design and optimize data flows from Databricks via mParticle into Braze to enhance customer journeys and audience activation.

• Develop automated frameworks for data quality, validation, governance, and monitoring to ensure reliable and compliant customer data.

• Empower CRM and marketing teams with access to trusted, activation-ready audiences and customer data with minimal latency.

• Manage customer consent, preferences, and PII throughout data flows, ensuring adherence to regulations such as GDPR and CCPA/CPRA.

• Diagnose and resolve pipeline and integration issues, enhancing reliability and performance while optimizing data consumption and infrastructure costs.

• Collaborate effectively with CRM, Martech, Product, Analytics, and Data teams within an Agile delivery framework.

• Offer technical guidance through code reviews, mentoring, documentation, and implementation of data engineering best practices.

• Assist in analytics and reporting use cases across Databricks and platforms including ThoughtSpot.

⛳️ Requirements

• Over 5 years of experience in Data Engineering, demonstrating strong hands-on capabilities in building production data pipelines.

• Extensive experience with Databricks, including Spark, PySpark, Structured Streaming, Delta Lake, and both batch and near real-time processing.

• Proficient in Python, SQL, and data modeling, with a background in designing scalable data architectures.

• Proven experience in integrating complex upstream or legacy data sources with downstream Martech, CDP, or customer engagement platforms.

• Strong grasp of customer data concepts such as identity resolution, unified customer profiles, deduplication, enrichment, customer events, and audience management.

• Familiarity with CDPs or customer engagement platforms, especially mParticle and/or Braze, and an understanding of customer data movement between these systems.

• Solid understanding of data quality, governance, privacy, consent management, and the handling of customer PII.

• Experience with cloud platforms like AWS, Azure, or GCP, along with knowledge of Git and CI/CD practices.

• Excellent problem-solving and communication abilities, capable of conveying technical concepts to non-technical stakeholders.

• Experience in an agency, consultancy, or client-facing role within Agile delivery teams.

• Proficiency in English at the C1 level.

• Databricks certification, such as Data Engineer Associate or Professional (Desirable).

• Experience with streaming, CDC, or event-driven technologies and patterns (Desirable).

• Familiarity with Delta Sharing, reverse ETL, or warehouse-native activation (Desirable).

• Experience with OneTrust or similar consent and preference management platforms (Desirable).

• Knowledge of analytics platforms like ThoughtSpot, Tableau, Power BI, or others (Desirable).

• Experience working with high-volume B2C or consumer datasets (Desirable).

• Background in managing customer data within Martech, CRM, loyalty, retail, travel, or other B2C contexts (Desirable).

• Proven ability to leverage AI tools to enhance engineering productivity and daily workflows (Desirable).

🏝️ Benefits

• Rocket‑Fuelled Growth – Tackle significant challenges, experience rapid learning, and get the opportunity to advance faster than anywhere else.

• A Culture That Actually Gives a Damn – A supportive and positive environment centered around individuals who genuinely want to see you succeed.

• A Global Crew – Work alongside brilliant colleagues from Europe, the US, and beyond.

• Remote‑First for Life – Enjoy the flexibility to work from wherever you feel most productive.

• Real Career Momentum – Benefit from clear advancement pathways, genuine ownership, and the opportunity to grow into your next chapter.

• Moments That Matter – Participate in meetups, events, and team experiences that make your journey memorable.