Lead AI ML Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop Navanta’s retrieval and verification processes across data systems, ensuring that every response is backed by well-documented queries and citations.

• Establish self-hosted open-weight models for serving and embeddings within each bank’s infrastructure or shared environments for Navanta; advance RAG to a specialized standard.

• Create the MCP tool layer that provides a limited, audited selection of read-only tools (metrics, documents, customer 360), with the potential to evolve into read/write tools managing large volumes of regulated, highly sensitive information.

• Construct and sustain the evaluation harness — including golden-question regression, groundedness and retrieval metrics, and explicit “I don’t know” behavior — and establish it as a release prerequisite.

• Implement LLM guardrails: ensure PII redaction in prompts and context, develop defenses against prompt injection, and set cost and row limits that align with regulatory security requirements.

• Collaborate with data teams to ensure that the model chooses governed metrics from the semantic layer instead of improvising SQL queries.

• Document the model’s architecture, evaluation methodologies, and guardrail controls to facilitate customer security assessments and readiness for audits.

• Monitor latency, cost, and quality trade-offs across different model versions and deployment configurations.

⛳️ Requirements

• 6–10+ years of software development experience, including 2–3+ years of delivering production LLM, RAG, or NLP systems utilized by actual users, rather than prototypes.

• A proven emphasis on accuracy and evaluation, extending beyond mere demonstrations.

• Proficient in Python and possess solid software engineering fundamentals.

• Comfortable working with self-hosted open-weight models and capable of analyzing latency, cost, and quality trade-offs.

🏝️ Benefits

• Typical office environment.

• Up to 20% travel time may be required.