Senior AI Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take charge of AI workflows from inception to completion, making decisions on development, deployment, measurement, and ongoing enhancement.

• Collaborate closely with Product teams, Backend Engineering, and customers to align objectives.

• Design systems utilizing agentic and LLM technologies that integrate extraction, retrieval, reasoning, and tool usage, ensuring clear human-in-the-loop interactions.

• Create design documentation, evaluate build-vs-buy decisions, select models, and establish success metrics.

• Develop production-quality LLM pipelines incorporating prompt and context design, structured outputs, validation, fallbacks, and confidence scoring.

• Seamlessly integrate AI services with Jeeves's backend by adhering to clear API contracts, implementing retries, ensuring graceful degradation, and maintaining individual customer data isolation.

• Oversee cost management and latency across various models.

• Construct evaluation sets and automate evaluations to detect regressions in response to changes in prompts, models, or data.

• Equip AI components with logging, tracing, dashboards, and alert systems.

• Maintain a comprehensive audit trail for AI decisions that comply with regulated financial product standards.

• Establish patterns, shared tools, and best practices for other engineers.

• Showcase effective utilization of coding agents and AI tools.

• Assess peers' AI system designs and disseminate insights.

⛳️ Requirements

• Over 7 years of professional experience in software engineering.

• Minimum of 2 years of experience in building and managing LLM or AI-powered systems in a production environment.

• Proven history of overseeing a significant system or workflow from initial scoping and design through to launch and iterative improvement, with minimal supervision.

• Hands-on experience in deploying LLM-powered applications using APIs like Anthropic, OpenAI, or similar, including structured outputs, error management, and evaluation processes.

• Expertise in designing agentic or multi-step AI workflows that incorporate tool usage, orchestration, and human review stages.

• Strong command of Python programming language.

• Solid foundation in backend technologies: REST APIs, PostgreSQL or comparable relational databases, asynchronous patterns, and experience with a leading cloud provider such as AWS, GCP, or Azure.

• Familiarity with observability in AI systems, including logging, tracing, dashboards, and quality assurance monitoring.

• Professional proficiency in English, both written and spoken.

🏝️ Benefits

• Access to AI-assisted tools that facilitate application reviews, interview scheduling, and note-taking.

• Options for accommodation or alternatives to AI-assisted processes are available upon request.