Data Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Collaborate with business stakeholders and product owners to gain insights into data product objectives, requirements, and success criteria.

• Design and implement scalable, secure, and high-performance data architecture on Microsoft Azure for both cloud-native and hybrid environments.

• Lead the development of data ingestion, transformation, and integration pipelines utilizing Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

• Manage data lakes and structured storage solutions with Azure Data Lake Storage Gen2.

• Integrate data from various source systems, such as ServiceNow and geospatial systems, through APIs, connectors, and custom scripts.

• Develop and maintain data models and semantic layers for reporting, analytics, machine learning, and downstream consumption.

• Build and optimize data workflows using Python and SQL to facilitate cleansing, enrichment, and advanced analytics.

• Design and expose secure data services and APIs via Azure API Management.

• Implement metadata management, data classification, and lineage tracking protocols.

• Ensure privacy and regulatory compliance through access controls, encryption, and data masking measures.

• Monitor and troubleshoot pipelines and integrations to ensure reliability, scalability, and performance.

• Utilize AI and automation tools for pipeline development, testing, monitoring, documentation, code generation, optimization, and review.

• Design and curate standardized, high-quality datasets for advanced analytics and prospective AI use cases.

• Provide operational support and continuous improvement for DRAS data products and platform solutions.

⛳️ Requirements

• A post-secondary degree, diploma, or certificate in Computer Science or a related field.

• 3 years of experience in designing data solutions for analytics-ready, trusted datasets.

• 4 years of experience utilizing version control systems.

• 3 years of experience with Azure services, including Storage, SQL, Synapse, and networking.

• 5 years of practical experience with Python and SQL in data engineering roles.

• 3 years of hands-on experience with Azure Databricks and Delta Lake.

• 1 year of experience using AI for code generation, data analysis, and automation.

• Nice to have: 6 years of experience in business requirements analysis related to data.

• Nice to have: 6 years of technical experience with Microsoft SQL Server.

• Nice to have: 2 years of experience with Microsoft Fabric.

• Nice to have: 1 year of experience building data products in a GOA Cloud Environment.

• Nice to have: 3 years of experience designing and integrating RESTful APIs.

• Nice to have: 3 years of experience with message queuing technologies.

• Nice to have: 5 years of experience working with cross-functional teams to develop software applications.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional growth and development.

• Flexible working arrangements to promote work-life balance.

• Access to cutting-edge technology and tools.

• Collaborative and inclusive work environment.