Senior Software Engineer – Data & ML Platform

This is a fully remote position, open to applicants in Canada.

📋 Description

• Take ownership of, operate, and enhance backend services deployed in Azure, encompassing serverless solutions, batch processes, and ML inference endpoints.

• Oversee deployments and ensure reliability across various environments, incorporating CI/CD, monitoring, alerting, incident response, and operational documentation.

• Manage and optimize cloud computing environments, focusing on autoscaling, container images, identity, and resource management.

• Enhance the reliability, scalability, and maintainability of production systems.

• Design and develop ETL/ELT pipelines from relational and document databases into datasets ready for analysis.

• Assist in creating a lakehouse-style analytical layer and its supporting infrastructure.

• Construct and uphold infrastructure-as-code across different environments utilizing Terraform or Bicep.

• Oversee cloud infrastructure, storage solutions, application hosting, identity and access management, Key Vault, and cost management.

• Implement monitoring, logging, data quality checks, and freshness alerts across data workflows.

• Develop internal services, APIs, and tools for developers.

• Create infrastructure and tools for ML and Operations Research experimentation, deployment, evaluation, and reproducibility.

• Convert research prototypes into dependable production services and workflows.

• Manage model packaging, versioning, deployment, and CI processes for ML and OR codebases.

• Establish automated evaluation and benchmarking pipelines for assessing model performance, drift, and reliability.

• Collaborate with Data Scientists and OR specialists to operationalize experiments.

• Set standards for code quality, automated testing, version control, and CI/CD practices.

• Conduct peer code reviews and advocate for scalable engineering practices.

• Work in tandem with Data Science, Operations Research, Product, and Engineering teams.

• Contribute to decisions regarding architecture, scalability, reliability, and performance.

• Translate both technical and business requirements into practical engineering solutions.

⛳️ Requirements

• Bachelor's or Master's degree in Computer Science, Software Engineering, Data Engineering, or a related discipline, or equivalent practical experience.

• Proficient programming skills in Python and SQL.

• Solid understanding of APIs, backend service architecture, and distributed systems.

• Experience in building and managing production data pipelines from start to finish, including retries, idempotency, backfills, orchestration, and freshness monitoring.

• Hands-on experience in designing and operating production services in Azure or another leading cloud platform, with a focus on Azure.

• Familiarity with serverless and batch compute, object storage, identity and access management, and monitoring tools.

• Experience with infrastructure-as-code tools such as Terraform, Bicep, or ARM.

• Strong knowledge of Git, CI/CD, automated testing, and contemporary software engineering methodologies.

• Comfortable working with ML or OR codebases and model artifacts, including reading, executing, packaging, and deploying them.

• Experience in managing live production systems and progressively enhancing an existing codebase.

• Preferred experience with Delta Lake, Parquet, lakehouse architectures, DuckDB, Polars, dbt, Azure Machine Learning, MLflow, DVC, Durable Functions, Airflow, Dagster, Prefect, Kubernetes, Azure data governance, security, DevOps/SRE, optimization, logistics, transportation, or large-scale ML systems.

🏝️ Benefits

• Opportunities for learning and career advancement.

• Health benefits.

• Paid time off.

• Equity options after the first year.

• A fun, close-knit team atmosphere.

• Celebrations of team culture and achievements.