Applied AI Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Transform scenario specifications, decision-making models, and assessment rubrics into prompts, application logic, and verifiable behavior.

• Identify missing requirements and technical limitations, and communicate them to Product, Engineering, and Subject Matter Experts (SMEs).

• Establish how AI models integrate with application services and authorized data sources, including APIs, tool interactions, data structures, validation, error handling, and allowed actions.

• Create prompts, structured outputs, and validation logic, ensuring invalid responses do not reach learners.

• Develop conversational runtimes that incorporate controlled personas, session states, and clear state transitions.

• Maintain separation between simulation behavior, assessment logic, and feedback to ensure that feedback does not influence assessment results.

• Execute grounding and retrieval from approved knowledge sources, providing source references where necessary.

• Manage code, prompts, schemas, and evaluation assets under version control, documenting them appropriately.

• Conduct bounded experiments with Product, focusing on clear questions and success metrics.

• Transition validated prototypes into production with Engineering while ensuring consistent behavior.

• Assist in creating technical controls that ensure AI features comply with accreditation standards, including issue flagging, review processes, disclosure, and logging.

⛳️ Requirements

• Proficient in prompt and context engineering: system prompts, few-shot examples, prompt chaining, routing, and breaking down complex tasks into manageable steps.

• Ability to produce structured outputs using JSON Schema, function calling, and validation layers, including retry and fallback mechanisms.

• Knowledge of RAG design: chunking, metadata filtering, hybrid search, reranking, and citation grounding.

• Experience in managing token budgets, prompt caching, context compaction, and long-session memory.

• Familiarity with agentic systems: tool usage, planner/executor patterns, multi-agent orchestration, state machines, and human-in-the-loop checkpoints.

• Skill in defining tool schemas, permission scopes, and action limits for agents.

• Experience with AI evaluation: creating golden datasets, rubric-based scoring, and regression testing calibrated against human raters.

• Ability to observe and trace LLM performance.

• Expertise in LLM security concerns: prompt injection, data leakage, jailbreaks, unsafe tool actions, and implementing guardrails.

• Comparative analysis of prompting, retrieval, and model options based on quality, latency, and cost.

• Proficiency in Python or TypeScript, including API integrations and automated testing.

• Familiarity with Git (e.g., Bitbucket), reproducible tests, and Continuous Integration (CI).

• Experience working with AI coding agents as a default practice.

• Ability to communicate technical trade-offs to non-technical team members.

• Sound judgment regarding privacy, human oversight, and the limitations of AI-generated assessments or advice.

• Direct experience in developing LLM applications with multi-step conversational workflows.

• Experience collaborating with subject-matter experts.

• Background in learning products, assessment, simulations, or regulated professional content is preferred.

🏝️ Benefits

• Fully remote, long-term opportunity.

• Collaboration with a widely distributed team.

• Agile-first team environment.