Senior Data Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Collaborate with business stakeholders and product owners to comprehend the objectives, requirements, and success criteria of data products.

• Design and implement a scalable, secure, and high-performance data architecture on Microsoft Azure within cloud-native and hybrid environments.

• Lead the development of data ingestion, transformation, and integration pipelines utilizing Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

• Manage data lakes and structured storage solutions in collaboration with the Data Architect using Azure Data Lake Storage Gen2.

• Integrate a variety of source systems, including ServiceNow and geospatial systems, through APIs, connectors, and custom scripts.

• Develop and maintain data models and semantic layers for operational reporting, analytics, machine learning, and downstream consumption.

• Create and optimize workflows in Python and SQL for data cleansing, enrichment, and advanced analytics within Azure Databricks.

• Design and expose secure data services and APIs via Azure API Management.

• Implement metadata management, data classification, and lineage tracking processes.

• Ensure compliance with privacy and regulatory requirements through role-based access controls, encryption, and data masking.

• Monitor and troubleshoot data pipelines and integrations to ensure reliability, scalability, and performance.

• Utilize AI and automation tools for the development, testing, monitoring, and documentation of pipelines.

• Leverage AI-assisted tools for code generation, optimization, and review.

• Design and curate standardized, high-quality datasets aimed at advanced analytics and future AI applications.

• Perform additional duties as required.

⛳️ Requirements

• Experience in designing scalable, secure, and high-performance data architectures on Microsoft Azure.

• Proficiency with Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

• Experience in managing Azure Data Lake Storage Gen2.

• Familiarity with integrating ServiceNow and geospatial systems using APIs, connectors, and custom scripts.

• Proficient in Python and SQL.

• Understanding of data modeling and the creation of semantic layers.

• Knowledge of Azure API Management.

• Awareness of metadata management, data classification, and data lineage practices.

• Knowledge of role-based access controls, encryption, and data masking techniques.

• Understanding of FOIP and GDPR compliance regulations.

• Ability to utilize AI and automation tools for data engineering workflows.

• Capacity to employ AI-assisted tools for code generation, optimization, and review.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health benefits including medical, dental, and vision coverage.

• Opportunities for professional development and continuous learning.

• Flexible work arrangements to promote work-life balance.

• A collaborative and inclusive work environment.