Senior Data Engineer

This is a fully remote position, open to applicants in Romania, +1 more country.

📋 Description

• Evaluate and redesign ingestion pipelines derived from operational and source-system data.

• Create and execute robust data mappings, lineage, dependencies, and controlled migration or cutover strategies.

• Develop and uphold comprehensive data reconciliation processes.

• Refactor and enhance data processing capabilities and hierarchy/roll-up features using PySpark.

• Introduce testing, versioning, metadata, observability, and deployment controls.

• Collaborate with business and technical subject matter experts (SMEs) to enhance data coverage.

• Evaluate GCP data pipeline options and assist with implementation.

• Explore opportunities to directly source mappings and reference data from systems like Microsoft Dynamics 365.

• Assist with changes to reporting hierarchy and filtering logic, including impact assessment, testing, and reconciliation.

• Develop and maintain automated data quality checks, monitoring, and operational controls.

• Facilitate the adoption of centralized reporting and data capabilities.

• Generate technical documentation, decision records, reconciliation evidence, variance explanations, and handover materials.

• Deliver results through a prioritized backlog and short iterative sprints with engineers and stakeholders.

⛳️ Requirements

• Over 5 years of professional experience in Data Engineering or related data roles, with substantial hands-on experience in building and supporting production data solutions.

• Strong hands-on experience with PySpark, SQL, and Databricks, including the development and maintenance of production-grade data pipelines.

• Practical experience with GCP data technologies and data pipelines.

• Experience in testing, optimizing, and providing operational support for data pipelines.

• Strong background in dimensional, master/reference, or hierarchical data modeling.

• Experience implementing intricate entity matching, data mappings, and roll-up logic with appropriate traceability.

• Practical experience in establishing automated data quality checks and reconciliation processes.

• Familiarity with observability, audit history, version control, and CI/CD within data engineering solutions.

• Experience in assessing source-system data and documenting data contracts, mappings, lineage, and dependencies.

• Experience managing controlled transitions of data pipelines or platforms.

• Ability to investigate data issues and clearly articulate variances in values, record counts, mappings, and hierarchy outcomes.

• Capability to work closely with platform teams, analysts, and business SMEs, translating ambiguous reporting and mapping requirements into tested engineering solutions.

• Excellent communication skills with the ability to produce clear technical documentation, decision records, variance explanations, and handover materials.

• Self-motivated and comfortable operating in a collaborative, iterative delivery environment.

• Preferred: Experience in Insurance, London Market, Reinsurance, broker, distribution, or policy-level data environments.

• Preferred: Familiarity with integrating Microsoft Dynamics 365 or similar operational source systems.

• Preferred: Experience with Power BI, including semantic models and reporting dependencies.

• Preferred: Knowledge of Slowly Changing Dimensions (SCDs) and historical hierarchy management.

• Preferred: Experience supporting complex master/reference data, entity matching, and hierarchy solutions.

• Preferred: Background in investigating historical data and reporting discrepancies.

• Preferred: Experience in supporting controlled data migrations, source-system transitions, or platform changes.

• Preferred: Experience working within distributed Agile teams and collaborating with UK-based business and technical stakeholders.

🏝️ Benefits

• Comprehensive health insurance and wellness programs.

• Flexible working hours and remote work options.

• Opportunities for professional growth and development.

• Collaborative and dynamic work environment.

• Competitive salary and performance-based bonuses.