AI Engineer, Agent/Platform Tracks

This is a fully remote position, open to applicants in India.

📋 Description

• Execute the specialized pharmacovigilance agents: compose system prompts, set model parameters, establish tool-use definitions, and delineate agent boundaries to guarantee accurate adverse event processing.

• Develop and refine prompt chains for each processing phase: document parsing, field extraction, MedDRA coding recommendations, causality assessment logic, narrative creation, and E2B(R3) output production.

• Create the deterministic rule engine layer: implement ICH E2B field validation checks, verify MedDRA hierarchy, and apply regulatory logic constraints that function alongside LLM outputs.

• Collaborate with the pharmacovigilance domain team to create and sustain evaluation datasets: annotated ground-truth cases, edge case libraries, and regression test suites.

• Design and manage Model Context Protocol (MCP) servers to standardize enterprise applications, APIs, databases, and services for AI agents.

• Implement secure MCP integrations, define tools, manage authentication, and conduct testing to facilitate reliable agent interactions with both internal and external systems.

• Conduct accuracy benchmarks, examine failure modes, and refine prompts and agent configurations to enhance performance against established thresholds.

• Establish the quality control agent's cross-verification logic: configure distinct Claude instances, develop comparison algorithms, and calibrate confidence scoring.

• Create human-in-the-loop feedback systems: reviewer interfaces for accept/modify/reject decisions, structured feedback collection, and feedback-to-prompt-improvement pipelines.

⛳️ Requirements

• Over 3 years of software engineering experience, including at least 1 year developing applications that utilize LLM APIs (Anthropic, OpenAI, or similar).

• Proficient in Python, with proven experience in production settings.

• Experience in building and evaluating NLP or LLM-based systems with quantifiable quality metrics.

• Strong problem-solving abilities and the capacity to work independently while collaborating with team members.

• Bachelor's degree in computer science or a related field, or equivalent professional experience.

• Excellent prompt engineering skills, with experience in writing and iterating system prompts, few-shot examples, chain-of-thought patterns, and structured output formats.

• Solid foundation in Python with practical experience in LLM orchestration frameworks such as LangChain, LangGraph, or similar tools.

• Experience in building evaluation pipelines for NLP or LLM outputs: precision/recall measurement, confusion matrices, and threshold tuning.

• Familiarity with AWS services including S3, Lambda, and basic IAM, with AWS Bedrock experience being advantageous.

• Exceptional written communication skills: ability to document prompt design decisions, evaluation outcomes, and agent behavior specifications for validation.

• A collaborative mindset and ability to effectively work with cross-functional teams, including domain experts and platform engineers.

🏝️ Benefits

• Flexibility, opportunities for growth, and an environment that fosters individuals to perform at their best.