Senior Data Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Lead the architecture and roadmap for the data lake.

• Design ingestion processes utilizing Zero-ETL, DMS, APIs, and device telemetry.

• Provide recommendations on how Redshift, Iceberg, and ClickHouse can accommodate analytical workloads.

• Establish data modeling standards for silver and gold datasets.

• Develop and maintain Glue/PySpark jobs, Lambdas, and orchestration with quality gates and thoroughly tested code.

• Take ownership of pipeline performance, reliability, cost management, partitioning, compaction, tuning, monitoring, and alerts.

• Classify data sources, minimize PII, mask lower environments, and maintain lineage documentation.

• Create and operate a framework that allows business users to prototype pipelines and automations securely.

• Collaborate with finance, sales, service, and product teams to develop reusable gold datasets.

• Prepare the platform for AI integration with governed datasets, least-privilege access, and audit trails.

• Utilize AI coding agents for building, testing, and documenting pipelines.

• Establish data engineering standards for Terraform, CircleCI CI/CD, code reviews, testing, and documentation.

• Provide support and mentorship to DevOps engineers in delivering data extracts and pipelines.

• Collaborate with the backend team on TypeScript, Lambda, DynamoDB, and Postgres data consumption.

⛳️ Requirements

• Extensive experience in building and operating AWS data platforms, preferably with a lakehouse or warehouse in production.

• Strong practical knowledge of Spark or Glue, Apache Iceberg or similar table formats, and a warehouse such as Redshift.

• Proficient in Python and SQL, with a proven track record of writing tested, maintainable pipeline code.

• Experience with CDC and replication (DMS, Zero-ETL) and integrating SaaS/ERP data through APIs.

• Solid background in data modeling and experience supporting BI and analytics consumers.

• Familiarity with infrastructure as code and CI/CD practices, ideally using Terraform with CircleCI or GitHub Actions.

• Strong understanding of data protection and governance, including classification, least privilege, and PII minimization.

• Daily engagement with AI coding agents; interest or experience in exposing data to AI tools such as MCP or semantic layers with governance.

• Ability to communicate clearly with both technical and business audiences.

• Experience in supporting or mentoring fellow engineers.

• Leadership experience in platform architecture or managing a small data workstream from start to finish is preferred.

• Familiarity with Amazon QuickSight, ClickHouse, Lake Formation, S3 Tables, and Glue 5 is a plus.

• Experience with Power BI is advantageous.

• Background in energy, IoT, or time-series data is desirable.

• Experience developing self-service or low-code data tools is favored.

• Familiarity with TypeScript and serverless architecture is desirable.

• Hands-on, pragmatic, curious, calm, methodical, and data-driven mindset; advocates for privacy, quality, and sustainability.

🏝️ Benefits

• Complimentary on-site EV charging.

• Private Medical Insurance.

• Cycle to Work Scheme.

• 25 days of holiday (increasing with tenure) plus Bank Holidays.

• Mental Health First Aiders available throughout the business.

• Enhanced employer pension contributions.

• Career development and continuous support.

• Employee empowerment and autonomy.

• Free on-site parking.

• A positive, purpose-driven work environment.

• EV salary sacrifice scheme.

• Birthday leave.

• Life Assurance - death in service.

• Long service awards.

• Enhanced maternity and paternity pay.

• Life Celebrations – vouchers for births and marriages.

• Employee Assistance Programme – a complimentary, 24/7 UK-based counseling helpline.