AI Engineer – Trust & Explainability, AI Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop tracing capabilities throughout the platform's gateway, orchestration, memory, and tool layers.

• Analyze actions among agents in multi-agent workflows, which include handoffs, parallel branches, and retries.

• Create trace views for developers and human-readable explanation layers.

• Generate explanation records for customers, along with confidence and provenance metadata, and summaries of agent dependencies.

• Collaborate with product engineers on the Document Request Agent and MLM agents.

• Assess, integrate, and enhance open-source frameworks for observability, tracing, and evaluation.

• Develop and manage evaluation tools, including golden dataset management, test runners, scoring pipelines, and regression reporting.

• Conduct model and prompt comparisons and document any changes.

• Create red-team and adversarial test suites for prompt injection, jailbreaks, tool misuse, and data exfiltration.

• Establish automated tests to validate tenant isolation across memory, retrieval, tool calls, and model context.

• Communicate red-team findings with Security Operations and incorporate threat models into platform testing.

• Engage in design discussions and code reviews.

• Assist in onboarding L1 AI Engineer team members and contribute to documentation efforts.

• Independently complete assigned features and bug fixes, write tests, monitor issues, and document implementation choices.

⛳️ Requirements

• Minimum of 3 years of professional software engineering experience, successfully delivering features in a production environment.

• Strong understanding of algorithms, data structures, and foundational software design principles.

• Skilled in Git, Docker, automated testing, and modern scripting languages.

• Regular daily usage of AI-assisted development tools.

• Bachelor's degree in Computer Science, Software Engineering, or equivalent experience.

• Practical experience in developing software that integrates large language models (LLM APIs, agent frameworks, RAG pipelines, or similar technologies).

• Proficient in Python with production experience; TypeScript is a plus.

• Hands-on experience in LLM observability and tracing, LLM evaluation and testing, agent frameworks, and multi-agent orchestration, or application security testing.

• Familiarity with distributed tracing or observability tools within a production system.

• Strong instincts for automated testing, including testing non-deterministic systems.

• Experience managing workloads on Azure or AWS, including identity and access management, networking, and secrets management.

• Preferred qualifications include experience with OpenTelemetry, GenAI semantic conventions, OpenLLMetry, LLM observability/evaluation tools, open-source AI projects, AWS Bedrock or Azure OpenAI, multi-tenant SaaS, regulated industries, and tools for developer-facing debugging or visualization.

• Must successfully complete MeridianLink's background check, credit check, and drug test as part of the offer process.

• Must have legal authorization to work in the United States or meet sponsorship requirements.

🏝️ Benefits

• Comprehensive insurance coverage (medical, dental, vision, life, and disability).

• Flexible paid time off.

• Paid holidays.

• 401(k) plan with company match.

• Opportunity for remote work.