Data Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Assume operational responsibility for usage and adoption pipelines as well as alerting mechanisms.

• Reconcile data from the product-usage tool for accuracy.

• Distinguish between development and production environments while enhancing CI/CD for consistent deployment and dependable rollback.

• Construct and sustain Bronze-to-Gold pipelines in Databricks, including Unity Catalog metric views and permissions/row-level-security models.

• Expand automated data quality checks across gold-layer tables and confirm outputs against existing Power BI reports.

• Collaborate on defining usage and adoption signals with Product and Post-Sales, focusing on upsell signals and ROI narratives.

• Address schema drift, inconsistent field naming, soft deletes, and incremental loads.

• Lead the integration of additional data sources, such as support tickets, audit logs, CRM, and other customer-facing systems.

• Assess feasibility, effort, cost, and implementation strategy for a future unified data platform.

• Create documented pipelines, access models for raw and transformed data, and a process for cross-department data requests.

• Serve as a point of contact for CoreView's Databricks environment.

• Implement automation and AI-driven tools, including monitoring and anomaly detection for metric fluctuations.

⛳️ Requirements

• 3 to 5 years of professional experience in developing and managing production data pipelines (ETL/ELT) within a commercial environment.

• Proficient in Databricks: Unity Catalog, Spark Declarative Pipelines, and metric views, including the underlying permissions/row-level-security model.

• Strong skills in SQL and Python applied to data engineering tasks, including building, testing, and supporting pipeline code in production.

• Familiarity with data transformation/orchestration tools such as Spark Declarative Pipelines, dbt, or similar solutions.

• Solid understanding of data modeling (dimensional/star-schema approaches) for analytical purposes.

• Experience with Git-based version control and CI/CD practices applied to data pipelines.

• Background in ingesting data from CRM systems, product usage metrics, telemetry, and other business systems.

• Experience with Power BI or a comparable BI tool (preferred).

• Previous involvement in implementing data governance, a metric catalog, or an access model (preferred).

• Experience in a B2B SaaS environment (preferred).

• Familiarity with AI coding assistants like GitHub Copilot, Cursor, or Claude (preferred).

• Excellent written and verbal communication skills.

• Ability to work independently and effectively implement, document, and operate according to established objectives.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional development and career advancement.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• Collaborative and inclusive company culture.