Applied AI Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of designing, building, and managing the LLM Gateway, serving as the sole pathway for production services to access model providers.

• Implement logging, cost attribution, rate limiting, failover mechanisms, and safeguards for protected health information (PHI).

• Eliminate direct provider API keys from services, ensuring model access is routed through the gateway.

• Oversee the approved catalog encompassing models, providers, endpoints, and model features.

• Document data retention policies, BAA coverage, version pinning, and log updates in decision records.

• Version prompts and model configurations concurrently with code.

• Conduct AI/ML evaluations as continuous integration release gates, facilitating staged rollouts and rollbacks.

• Offer self-service templates for sandbox and staging environments while controlling access to production.

• Transform validated AI/ML prototypes, evaluation harnesses, and acceptance criteria into scalable, observable production services.

• Provide insights on model costs, latency, and availability.

• Integrate LLM services with OpenTelemetry to track latency, failures, token usage, costs, and output quality.

• Monitor evaluation drift, participate in on-call duties, and lead incident response for LLM-supported services.

• Generate audit documentation in collaboration with Security and Compliance for SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11 compliance.

• Work in conjunction with the CTO, platform engineering, machine learning teams, researchers, domain experts, sales, and delivery teams.

⛳️ Requirements

• Minimum of 5 years of experience in backend software engineering, particularly with multi-tenant SaaS services and APIs in a production environment.

• Hands-on experience in building with commercial LLM APIs, focusing on prompt management, structured outputs, and tool integration.

• Proven expertise in designing evaluation pipelines for LLM outputs and integrating them into CI/CD processes as release gates.

• Strong proficiency in Python.

• Familiarity with AWS, containerization (ECS or similar), and infrastructure as code practices (e.g., Pulumi, Terraform, or CDK).

• Experience with observability tools such as OpenTelemetry, Datadog, or similar platforms.

• Knowledge of version control systems, feature flags, CI, and issue tracking tools like Linear.

• Understanding of application security practices, including secrets management, access control, and the handling of sensitive data.

• Preferred: experience in regulated industries such as healthcare or life sciences, particularly with systems validated for HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11.

• Preferred: experience in developing an internal developer platform or paved road utilized by other teams.

• Preferred: background in LLM-as-judge evaluation, red teaming, or conversational analytics.

• Preferred: experience in training or fine-tuning models from scratch.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible working hours and the option for remote work.

• Opportunities for professional development and continuous learning.

• A supportive and collaborative work environment.