AI Engineer – Applied AI

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop, assess, and fortify the orchestration, context, tools, and evaluations for the AI agent.

• Take ownership of complete slices, encompassing prompt and context architecture, retrieval across policy and coverage data, tool and API integrations with core claims systems, and human handoff procedures.

• Construct and operate the evaluation loop, which includes golden sets, trajectory assessments, LLM-as-judge evaluations, and error analysis based on actual production call traces.

• Ensure that evaluations serve as gatekeepers for releases.

• Enhance containment and resolution rates while maintaining accuracy.

• Implement instrumentation in the system to track variations in containment, resolution, and accuracy.

• Work on the voice path, focusing on aspects such as turn-taking, interruptions, latency budgets, and the LiveKit and SIP telephony layers.

• Participate in carrier calls and review recordings to pinpoint conversation issues and potential enhancements.

• Assist in determining the next actions for the agent and making decisions about launching an MVP or decelerating for accuracy in claims workflow.

⛳️ Requirements

• Proven experience in building and deploying AI applications, particularly an LLM application utilized by real users.

• Proficiency in LLMs, context engineering, RAG, and agentic workflows.

• Strong foundation in ML and deep learning.

• Familiarity with structured evaluations, error analysis, and statistical metrics.

• Understanding of software engineering fundamentals, including trade-offs related to cost, latency, reliability, and scalability.

• Experience in designing data stores and distributed systems.

• Capability to write tests effectively.

• Dedication to ensuring security and privacy concerning claims data, recorded calls, and PII.

• Background in advocating for CI/CD process automation.

• Daily use of coding agents.

• Possess product sense and sufficient insurance context to influence agent behavior and priorities.

• Openness to continuous learning as the field evolves.

• Nice to have: experience with real-time or streaming voice systems (ASR, TTS, telephony, SBC/SIPREC).

• Nice to have: experience in insurance, claims, or any regulated domain.

• Nice to have: experience with internal evaluation or observability tools.

• Nice to have: knowledge of machine learning principles or experience with experimenting, training, and deploying models in production.

• Nice to have: exposure to P&C insurance or Guidewire products and services.

• Nice to have: familiarity with prompt or pipeline optimization frameworks such as DSPy or GEPA applied to a production environment.

🏝️ Benefits

• Health insurance.

• Dental insurance.

• Vision insurance.

• Paid time off.

• Company-sponsored retirement plan.

• Some roles may qualify for the annual company bonus plan.

• Some roles may be eligible for commissions.

• Some roles may qualify for long-term incentive awards.

• Disability accommodations available during the hiring process.

• Appeals process for denied accommodations or non-selection decisions.