Software Engineer, AI Systems

This is a fully remote position, open to applicants in Canada.

📋 Description

• Develop and manage multi-step LLM pipelines that coordinate model calls, tool interactions, graph queries, retrieval processes, quality control, and transitions between specialist agents.

• Enhance Haven’s coordinated agent team and orchestration framework, spanning incident evidence collection, analysis, review, and enterprise learning.

• Create the context layer that integrates Neo4j graph traversal, vector search, and hybrid retrieval techniques.

• Construct evaluation datasets, scoring systems, regression suites, model comparisons, human-label loops, and quality attribution for each stage.

• Execute tracing, tool-call audits, cost and latency monitoring, failure management, and quality dashboard implementation.

• Identify loops, hallucinations, and unnoticed model drift prior to customer detection.

• Choose models from OpenAI, Anthropic, and Google based on specific task requirements.

• Collaborate with product and knowledge engineering teams to help define the AI roadmap.

• Report directly to the CTO and engage in various responsibilities as part of the founding team.

⛳️ Requirements

• Experience in AI or ML engineering, particularly in deploying LLM systems relied upon by real users.

• Practical experience in building and troubleshooting multi-step, tool-calling workflows using LangGraph, LangChain, or similar frameworks.

• A consistent approach to LLM evaluation that includes representative datasets, regression testing, LLM-as-judge techniques, or human review loops.

• Proven ability to assemble context for LLMs and make informed decisions regarding what to retrieve, how much, and why.

• Demonstrated history of managing systems from deployment through monitoring and incident resolution.

• Experience in diagnosing and rectifying production failures or regressions.

• Ability to work across various model providers and articulate trade-offs in quality, latency, cost, context, and operational risk.

• Proficiency with Neo4j and Cypher, or a similar graph database, with a quick learning curve for graph data modeling; strongly preferred.

• Strong skills in Python.

• Experience in production with FastAPI, asynchronous services, testing, observability, and maintainable interfaces.

• In-depth graph experience, including Cypher, schema evolution, MERGE patterns, embeddings, and managing a live knowledge graph; nice to have.

• Experience in enterprise AI security; nice to have.

• Familiarity with Azure and hybrid search; nice to have.

• Experience with platforms like Azure AI Search, Pinecone, MongoDB Atlas, pgvector, Elasticsearch, or similar; nice to have.

• Previous experience in enterprise-level B2B SaaS; strongly preferred.

🏝️ Benefits

• Small team with significant ownership opportunities.

• Direct feedback from safety teams and visible impact on customers.

• Opportunity to collaborate closely with the CTO, product, and knowledge engineering teams.

• Involvement in significant technical decision-making with rapid customer impact.

• Engaging reasoning challenges in a safety-critical domain.

• A culture centered around evaluation, focusing on measurable, observable, and improvable quality.