LLM Systems – AI Agent Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Develop and enhance production AI agents utilizing foundation models, specifically AWS Bedrock.

• Create and improve orchestration for production AI agents.

• Implement context engineering within production LLM and agent systems.

• Construct and maintain evaluation datasets and pipelines, which include tool selection, trajectory assessment, and LLM-as-judge evaluations.

• Establish and oversee production observability and monitoring for LLM and agent systems.

• Execute tracing and instrumentation for production LLM systems.

• Collaborate with the engineer leading the AI-agenting function as the first dedicated hire.

• Assist in deciding whether to keep the custom orchestration layer or transition to a production framework like LangGraph or LangChain.

• Participate as a member of a new three-person product team alongside the AI function lead and a Full Stack Engineer.

• Function as an individual contributor without managing others.

• Take immediate ownership of measurable deliverables upon onboarding.

• Deliver comparable AI/agent engineering work according to roadmap schedules.

⛳️ Requirements

• Minimum of 2 years of experience in building production LLM agents, encompassing tool-calling agent loops, streaming, context management, structured outputs, and orchestration frameworks.

• Experience in production with an agentic framework such as LangGraph, LangChain, or custom orchestration.

• At least 1.5 years of experience in evaluation-driven development, including evaluation datasets and pipelines that cover tool selection, trajectory evaluation, and LLM-as-judge.

• Over 1 year of experience with LLM observability, tracing, and instrumentation using tools like Langfuse, OpenTelemetry, or similar technologies.

• Authentic experience in production agent observability; direct hands-on experience with Langfuse is highly preferred.

• More than 1 year of experience in LLM cost optimization, including prompt caching, model selection and routing, and LLM FinOps.

• Over 5 years of backend engineering expertise, including TypeScript/Node, Postgres, and serverless AWS.

• Demonstrated ability to deliver similar work within comparable timelines.

• Experience in deploying agentic AI systems to production, addressing specific failure modes and mitigation strategies, and working end-to-end across the AI stack.

• Preferred: 1+ year of AWS Bedrock experience or relevant production experience with OpenAI, Anthropic API, Azure OpenAI, or Vertex AI.

• Preferred: 1+ year of experience with AI safety and guardrails, including prompt-injection screening, output validation, and managing untrusted input.

• Preferred: 6+ months of familiarity with MCP and multi-agent patterns.

• Preferred: knowledge of geospatial data.

🏝️ Benefits

• Fixed shifts: 12:00 PM - 9:30 PM IST (Summer) | 1:00 PM - 10:30 PM IST (Winter).

• No weekend work required.

• Laptop provided from Day 1.

• Comprehensive medical insurance offered from Day 1.

• Access to mentorship, community, and forums for idea sharing.

• Opportunity for long-term career growth where contributions are recognized.