Machine Learning Engineering Manager – Evaluations

This is a fully remote position, open to applicants in Austria.

📋 Description

• You will oversee and expand a team of exceptional Machine Learning Engineers and Research Scientists based in the EU, who are pioneering the future of AI at scale.

• Your primary focus will be on establishing strategic technical direction, mentoring team members to deliver impactful engineering solutions, and ensuring the deployment of robust, scalable ML systems in production.

• You will advocate for engineering excellence and measurable outcomes, connecting foundational model capabilities with real-world deployment across Canva's platform.

• Providing coaching and mentorship to a high-performing team of Machine Learning Engineers and Research Scientists.

• Taking ownership of the evaluation infrastructure - designing, building, and maintaining robust evaluation systems, quality metrics, safety monitoring, red-teaming, and competitive benchmarking to ensure enterprise readiness and user satisfaction at scale.

• Developing automated metrics that accurately predict human aesthetic judgment across aspects such as visual hierarchy, layout coherence, typography, and brand alignment.

• Offering guidance on human evaluation pipelines and facilitating the connection between user signals and model enhancements.

• Establishing technical strategy in alignment with Canva's AI and product objectives.

• Directing engineering efforts across model deployment, evaluation infrastructure, and production systems.

• Collaborating cross-functionally to ensure ML capabilities effectively translate into reliable product impact.

⛳️ Requirements

• Experienced in leading machine learning engineering teams, with a proven track record in coaching and delivering production systems.

• Possess deep expertise in deploying and scaling generative models (Diffusion, GANs, VAEs, LLMs) in production settings, particularly with a strong emphasis on visual models (image, video, design).

• Bring practical experience in building ML infrastructure, evaluation pipelines, and monitoring systems at scale.

• Skilled in developing data-driven evaluation methodologies, transforming user analytics and production metrics into clear, actionable insights.

• Strong systems design capabilities and experience with MLOps, model serving, and ensuring production reliability.

• Familiarity with visual quality assessment, aesthetic modeling, or human preference learning is a plus—especially if you have addressed the disparity between automated metrics and human ratings.

• Well-versed in design principles (hierarchy, balance, typography, color theory) enough to operationalize them as measurable signals.

• Excel in collaborative settings and communicate effectively with both technical and non-technical audiences.

• Stay updated with current SOTA research trends and engineering best practices, driven by a passion for continuous learning.

🏝️ Benefits

• Equity packages - we want our success to be yours too.

• Inclusive parental leave policy that supports all parents and caregivers.

• An annual Vibe & Thrive allowance to promote your wellbeing, social connections, home office setup, and more.

• Flexible leave options that empower you to be a force for good, take time to recharge, and support your personal needs.