Generative AI Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Create comprehensive AI solutions utilizing Dataiku's platform, including Dataiku Agent Hub, Prompt Studio, LLM Mesh, Knowledge Banks (Vector Stores), and when necessary, Python-based frameworks.

• Develop and manage multi-agent systems through Dataiku's Visual Agents (both simple and structured) and relevant code-based frameworks (LangGraph, CrewAI, Claude Agent SDK, OpenAI Agents SDK).

• Optimize and integrate LLM APIs from various providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, open-source models via Dataiku's LLM Mesh), employing model routing strategies to achieve an optimal balance of cost, latency, and quality.

• Execute Retrieval-Augmented Generation (RAG) pipelines, including agentic RAG and GraphRAG, utilizing Dataiku's Knowledge Banks with capabilities for reranking, dynamic filtering, and document extraction.

• Collaborate primarily with the "Revenue" organization, including Sales, BDR, Customer Success, Solutions Engineering, Professional Services, Sales Operations, and Marketing (approximately 75% of the role), while also extending proven solutions throughout the organization (approximately 25%).

• Engage with stakeholders to collect business requirements, going beyond to identify the core user pain these requirements signify, and design solutions that meet both the explicit need and the underlying problem.

• Manage projects from inception to completion, overseeing requirements gathering, solution design, development, deployment, and handover.

• Create autonomous and semi-autonomous AI agents utilizing Dataiku's Agent Builder, custom Python-based architectures (LangGraph, CrewAI, Claude Agent SDK, etc.), or a combination of both, exercising discretion on when to utilize platform features versus custom solutions.

• Develop Agent Tools that extend beyond documented examples, including custom API integrations, data retrieval modules, decision-making logic, and automated workflows, moving beyond standard patterns to deliver tailored solutions for specific business challenges.

• Construct, publish, and utilize MCP (Model Context Protocol) servers to facilitate agent-to-tool integration across systems, including designing custom MCP servers when necessary.

• Establish evaluation and monitoring strategies for agent systems, integrating Dataiku's built-in capabilities with custom instrumentation to assess reliability, accuracy, cost, and business impact in production.

• Design and uphold evaluation frameworks (evals) for LLM-based systems, measuring accuracy, latency, cost, and reliability in a production environment.

• Comply with data governance, security, and regulatory compliance requirements (awareness of EU AI Act, responsible AI practices) for all AI solutions.

• Utilize Dataiku's Cost Guard and Quality Guard features to control LLM expenses, enforce usage policies, and sustain output quality standards.

• Collaborate closely with analytics and data engineering teams to manage metadata concerning reference datasets for LLM consumption.

• Develop front-end user interfaces for AI applications using HTML, CSS, and JavaScript, within Dataiku's webapps framework, Dataiku Answers for chat-based interfaces, or standalone applications built with Vue.js and Node.js.

• Participate in UX design efforts, ensuring that internal stakeholders find AI solutions user-friendly and responsive.

• Provide feedback to the product development team to enhance the platform.

• Stay updated with the fast-evolving landscape of AI engineering, agent frameworks, model capabilities, evaluation techniques, governance requirements, and tools like MCP and A2A protocols.

⛳️ Requirements

• Strong Python skills are essential, including familiarity with standard data science and AI engineering libraries.

• Hands-on experience building agentic AI systems, orchestrating multi-agent environments, tool chaining, autonomous decision-making, and deploying AI agents in production is required.

• Experience with contemporary agent orchestration frameworks (LangGraph, CrewAI, Claude Agent SDK, OpenAI Agents SDK, or comparable); familiarity with LangChain is relevant but not sufficient on its own.

• Understanding of RAG architectures (vector databases, embedding strategies, agentic RAG, GraphRAG) and knowledge of when to implement each approach is necessary.

• Familiarity with MCP (Model Context Protocol) for agent-to-tool integration or a proven ability to quickly learn new integration standards is preferred.

• Experience with structured outputs, function/tool calling, and prompt engineering across multiple LLM providers is beneficial.

• Knowledge of web development basics (HTML, CSS, JavaScript); experience with Vue.js and Node.js is a plus.

• Exposure to AI evaluation practices, including building evals, monitoring model/agent performance in production, and iterating based on metrics is advantageous.

• Comfort with AI-assisted development tools (GitHub Copilot, Cursor, Claude Code, or similar) is required.

• Familiarity with Dataiku is a bonus.

🏝️ Benefits

• Opportunities for professional development

• Options for remote work