Senior Engineer, Data Platform

This is a fully remote position, open to applicants in Canada.

📋 Description

• Assist in the conceptualization and execution of data ingestion pipelines and infrastructure.

• Aid in the effective transition of legacy data lifecycle management to a new platform, ensuring minimal disruption for current data users.

• Develop and uphold Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for the new ingestion and data platform, utilizing dashboards and alerts to monitor performance.

• Create a versatile data storage layer that accommodates various use cases, including transactional, analytical, and machine learning tasks.

• Implement thorough monitoring, observability, and incident response procedures for all event data pipelines and services.

• Work in collaboration with product engineering, analytics, and machine learning teams to establish contracts, functional requirements, and standards.

• Design and execute a semantic metadata layer that categorizes and labels data assets across both products, facilitating consistent data discovery, exposure policies, and identification of opportunities for cross-product data reuse.

• Architect and deliver a multi-tenant data model that ensures secure data sharing and isolation among clients, with controls aimed at meeting regulatory and governmental compliance standards.

⛳️ Requirements

• Profound knowledge of data governance principles and data lifecycle management, encompassing data quality, lineage, retention, and access control.

• Strong understanding of database backend technologies (OLAP vs. OLTP) and their appropriate applications, with practical experience in utilizing data warehouse technologies (Databricks, Clickhouse, Redshift, etc.) at a production scale.

• Strong proficiency in SQL, capable of writing and optimizing queries effectively.

• Expertise in constructing complex systems, identifying core primitives, and applying them to address evolving business needs and future roadmap objectives.

• Experience in designing and managing data systems on a major cloud provider (AWS or GCP) and operating within a multi-cloud deployment environment.

• Proficient in containerization and orchestration technologies, including Docker and Kubernetes.

• Demonstrated ability to integrate systems, connecting legacy platforms with modern architectures through well-crafted interfaces and migration strategies.

• Effective communicator adept at facilitating system design discussions, documenting architectural decisions, and collaborating across team boundaries.

• (Nice to have) Familiarity with Elixir for developing concurrent, fault-tolerant data services.

• (Nice to have) Experience with data pipeline and streaming technologies such as Airflow, Kafka, Apache Flink, or similar.

• (Nice to have) Practical experience with columnar/OLAP databases like Databricks or ClickHouse.

• Experience in designing and managing data systems on AWS; GCP experience is an advantage.

• Strong collaboration and communication skills, comfortable leading design discussions, drafting technical specifications, and working across team boundaries.

• Experience with Infrastructure-as-Code (IaC) tools such as Terraform, CloudFormation, or similar.

🏝️ Benefits

• A comprehensive range of medical, financial, and/or additional benefits.