Staff Machine Learning Platform Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design and manage ML infrastructure, encompassing workspaces, clusters, jobs, and workflows.

• Productionize ML workloads utilizing Spark, Delta Lake, MLflow, and Databricks Workflows.

• Educate data scientists on utilizing the ML platform and transitioning models from notebooks to production.

• Implement Unity Catalog for effective data governance, lineage tracking, access control, and secure multi-tenant operations.

• Develop CI/CD pipelines for ML leveraging Terraform and Git-based workflows such as GitHub Actions.

• Enhance performance, reliability, and cost-efficiency across training and inference workloads.

• Configure IAM and RBAC for managing sensitive datasets.

• Establish observability for data quality, model performance, and overall platform health.

• Create and maintain technical documentation for the ML Platform.

⛳️ Requirements

• Over 8 years of experience in constructing production ML or data platforms.

• A degree in Computer Science, Engineering, Statistics, or a related technical discipline; a graduate degree is preferred.

• Strong practical knowledge of Databricks, Spark, Delta Lake, and MLflow.

• Proficiency in Python, SQL, and principles of distributed systems.

• Experience with cloud platforms and infrastructure-as-code methodologies.

• Familiarity with MLOps best practices, encompassing CI/CD, monitoring, reproducibility, and security.

• Experience in supporting multiple ML teams within a shared platform environment.

• Capability to take ownership of unresolved issues and acquire necessary knowledge to finalize tasks.

• Proficiency with relevant technologies such as Kotlin, PyTorch, Kafka, Snowflake, Airflow, AWS, Kubernetes, Docker, and Terraform.

🏝️ Benefits

• Equity.

• Comprehensive benefits.

• Competitive pay.

• Access to the latest enterprise AI tools.

• Flexible remote work option for candidates residing in Ontario, Canada.