Senior Data Engineer

Posted 6 days ago

This is a fully remote position, open to applicants in Canada.

📋 Description

• Develop and implement robust big data pipelines for both structured and unstructured data across various geographical regions.

• Assist healthcare organizations in achieving their business objectives by leveraging advanced data ingestion technologies, cloud services, and DevOps methodologies.

• Partner with architects from Cloud Engineering, Software Engineering, and ML Engineering.

• Create platforms, solutions, and applications that respond to healthcare trends such as digital diagnosis, software as a medical product (SaMD), and AI marketplaces.

• Build migration, storage, and processing utilities utilizing Azure Synapse Analytics.

• Formulate and execute cloud migration strategies for both cloud-based and on-premise applications.

• Identify and resolve issues within distributed systems in the Azure ecosystem.

• Develop tools to ingest and process multiple terabytes or petabytes of data daily.

• Enhance performance and cost-efficiency within Azure Synapse.

• Create and implement storage and compute solutions for large clients using Azure Synapse.

• Collaborate with Infrastructure, Network, Engineering, DevOps, SiteOps, and cloud customer teams.

• Develop automation, testing, monitoring, administration, and data operations tools across Azure Synapse workspaces and cloud clusters.

• Guide technology teams and mentor junior staff members.


⛳️ Requirements

• Over 6 years of experience (Experience Level).

• More than 3 years of hands-on experience in Data Engineering.

• In-depth knowledge of data structures, distributed systems, data modeling, and databases.

• Strong command of SQL and practical experience with Python/PySpark.

• Extensive experience in developing Big Data solutions for migration, storage, and processing.

• Proven experience in implementing solutions on cloud platforms.

• Experience in building and maintaining large-scale systems in a production environment.

• Solid experience in developing ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines.

• Profound expertise in Azure, particularly Azure Synapse Analytics, including SQL Pools, Spark Pools, Data Explorer, and Pipelines.

• Familiarity with Azure Data Lake Storage (ADLS Gen2).

• Strong background in Apache Spark, especially with Azure Synapse Spark Pools.

• Proficiency in Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other Azure data services.

• Experience with Azure Data Factory, Apache Airflow, or similar orchestration frameworks.

• Knowledge of Azure Event Hubs, Azure Service Bus, or Kafka.

• Capability to lead technology teams and provide mentorship and support.

• Experience managing multiple large projects.

• Deep understanding of Agile methodologies.

• Effective communication skills with both technical and non-technical stakeholders.

• Experience in the insurance data/domain is a plus.

• Familiarity with Apache Airflow, Databricks, dbt, Microsoft Fabric, or Snowflake is advantageous.

• Knowledge of Kubernetes/AKS is beneficial.

• Understanding of Terraform/Infrastructure as Code and Azure DevOps is desirable.

• Experience in data governance, data quality, lineage, and metadata management is a plus.

• Skills in team management and mentoring.

• Ability to mentor and guide junior team members.

• Capacity to clearly communicate complex subjects and technical plans.

• Proficiency in writing clear reports accessible to diverse audiences.


🏝️ Benefits

• Opportunities for upskilling and professional development.

• Exposure to pioneering AI, ML, data, and cloud technologies.

• Opportunities to collaborate with Fortune 500 companies.

• Work within a research-driven organization with over 60 patents filed.

• Collaborate with motivated and talented colleagues.

• Opportunities to make a significant impact in a rapidly growing AI-first digital engineering company.

People also viewed

Trax Technologies11 hours ago

Data Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Fiserv11 hours ago

Senior Snowflake Data Platform and Artificial Intelligence Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Ontrac Solutions11 hours ago

Staff Data Engineer

US flagUnited States OnlyFreelanceData Engineer
ApplyView job
General Dynamics Information Technology11 hours ago

ICAM Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology11 hours ago

Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology11 hours ago

Senior Data Engineer

US flagUnited States OnlyFull-timeData Engineer$102k – $138k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers