
Senior Data Engineer
Posted 6 days ago

Posted 6 days ago
This is a fully remote position, open to applicants in Canada.
• Develop and implement robust big data pipelines for both structured and unstructured data across various geographical regions.
• Assist healthcare organizations in achieving their business objectives by leveraging advanced data ingestion technologies, cloud services, and DevOps methodologies.
• Partner with architects from Cloud Engineering, Software Engineering, and ML Engineering.
• Create platforms, solutions, and applications that respond to healthcare trends such as digital diagnosis, software as a medical product (SaMD), and AI marketplaces.
• Build migration, storage, and processing utilities utilizing Azure Synapse Analytics.
• Formulate and execute cloud migration strategies for both cloud-based and on-premise applications.
• Identify and resolve issues within distributed systems in the Azure ecosystem.
• Develop tools to ingest and process multiple terabytes or petabytes of data daily.
• Enhance performance and cost-efficiency within Azure Synapse.
• Create and implement storage and compute solutions for large clients using Azure Synapse.
• Collaborate with Infrastructure, Network, Engineering, DevOps, SiteOps, and cloud customer teams.
• Develop automation, testing, monitoring, administration, and data operations tools across Azure Synapse workspaces and cloud clusters.
• Guide technology teams and mentor junior staff members.
• Over 6 years of experience (Experience Level).
• More than 3 years of hands-on experience in Data Engineering.
• In-depth knowledge of data structures, distributed systems, data modeling, and databases.
• Strong command of SQL and practical experience with Python/PySpark.
• Extensive experience in developing Big Data solutions for migration, storage, and processing.
• Proven experience in implementing solutions on cloud platforms.
• Experience in building and maintaining large-scale systems in a production environment.
• Solid experience in developing ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines.
• Profound expertise in Azure, particularly Azure Synapse Analytics, including SQL Pools, Spark Pools, Data Explorer, and Pipelines.
• Familiarity with Azure Data Lake Storage (ADLS Gen2).
• Strong background in Apache Spark, especially with Azure Synapse Spark Pools.
• Proficiency in Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other Azure data services.
• Experience with Azure Data Factory, Apache Airflow, or similar orchestration frameworks.
• Knowledge of Azure Event Hubs, Azure Service Bus, or Kafka.
• Capability to lead technology teams and provide mentorship and support.
• Experience managing multiple large projects.
• Deep understanding of Agile methodologies.
• Effective communication skills with both technical and non-technical stakeholders.
• Experience in the insurance data/domain is a plus.
• Familiarity with Apache Airflow, Databricks, dbt, Microsoft Fabric, or Snowflake is advantageous.
• Knowledge of Kubernetes/AKS is beneficial.
• Understanding of Terraform/Infrastructure as Code and Azure DevOps is desirable.
• Experience in data governance, data quality, lineage, and metadata management is a plus.
• Skills in team management and mentoring.
• Ability to mentor and guide junior team members.
• Capacity to clearly communicate complex subjects and technical plans.
• Proficiency in writing clear reports accessible to diverse audiences.
• Opportunities for upskilling and professional development.
• Exposure to pioneering AI, ML, data, and cloud technologies.
• Opportunities to collaborate with Fortune 500 companies.
• Work within a research-driven organization with over 60 patents filed.
• Collaborate with motivated and talented colleagues.
• Opportunities to make a significant impact in a rapidly growing AI-first digital engineering company.
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.