Senior Data Engineer – Microsoft Fabric

This is a fully remote position, open to applicants in Canada.

📋 Description

• Architect, construct, and manage the Bronze → Silver → Gold lakehouse framework within Microsoft Fabric, adhering to medallion design principles for the incremental enhancement of data.

• Create resilient, idempotent data pipelines featuring data quality assessments and validations at each tier boundary.

• Establish and oversee data products, data models, ownership delineations, SLAs, and documentation pertaining to the Data Hub platform.

• Collaborate with architects and stakeholders to define and adhere to data architecture standards throughout the platform.

• Guarantee data accuracy, completeness, and consistency across various data sources, transformation layers, and consumer interfaces.

• Detect, diagnose, and resolve data-related challenges within pipelines, schemas, transformations, and downstream utilization.

• Work closely with Data Scientists, Data Analysts, and business units to comprehend data needs and provide tailored data products.

• Offer technical advice and assistance on data-related issues among teams.

• Record data flows, schemas, transformations, pipelines, and operational processes to enhance maintainability and team scalability.

⛳️ Requirements

• Senior-level candidate with demonstrated hands-on experience in data engineering and technical stewardship of production data platforms.

• Capable of working autonomously and assuming technical responsibility.

• Excellent communication abilities and experience collaborating within cross-functional teams.

• Extensive expertise in PySpark, Spark SQL, and Delta Lake, covering partitioning strategies, MERGE patterns, and the trade-offs between batch and streaming.

• Practical experience with Microsoft Fabric or Azure Databricks.

• Hands-on knowledge of Azure data platforms, including Azure Synapse, Microsoft Fabric, and active development experience in Databricks for pipeline creation and optimization.

• Strong grasp of event-driven data ingestion patterns; familiarity with Azure Event Hubs is a significant advantage.

• Comprehensive understanding of data modeling and database design tailored for analytics and business use.

• Experience with version control and CI/CD methodologies using GitHub Actions, which is prevalent for Azure and Fabric-related deployment workflows.

• Solid understanding of Azure cloud services and data platform offerings.

• Capability to conceptualize data products and domain ownership, while defining clear interfaces between platform teams and their consumers.

• Strong operational mindset for managing and supporting pipelines in a production setting.

🏝️ Benefits

• Excellent team atmosphere

• Options for remote work