Software Engineering SMTS / LMTS – AI Core

This is a fully remote position, open to applicants in California.

📋 Description

• Design, develop, and manage retrieval systems utilizing RAG and hybrid search methodologies.

• Take ownership of retrieval projects from initial scoping and design to launch, monitoring, and iterative improvements.

• Lead the technical design of frameworks and platform components for building production agents.

• Create multi-agent orchestration, shared primitives for prompts and tools, as well as tracing and evaluation infrastructure.

• Identify and troubleshoot answer quality issues by differentiating between retrieval and generation problems.

• Establish and monitor quality metrics while spearheading improvement initiatives.

• Design models for document ingestion, segmentation, labeling, and retrieval for knowledge bases and call transcripts.

• Develop practices for evaluation, observability, rollout, offline evaluation, online monitoring, and regression detection.

• Assess solution paths and decide when to trust, redirect, or override the outputs of models and agents.

• Define and monitor performance and scaling objectives for retrieval, indexing, LLM provider usage, and agent orchestration.

• Ensure the reliability and performance of the systems under your ownership.

• Participate in on-call duties and lead incident resolution efforts.

• Implement coding agents for tasks involving implementation, testing, refactoring, and documentation.

• Develop reusable workflows, patterns, and tools to enhance team productivity.

• Collaborate with product, design, and engineering leaders on roadmaps, trade-offs, requirements, and execution strategies.

• Mentor engineers, conduct code reviews, and enhance engineering standards, tools, and processes.

⛳️ Requirements

• Demonstrated experience delivering production AI or backend systems in a high-ownership setting, with recent emphasis on LLM-powered applications.

• Proficiency in developing retrieval systems using RAG and hybrid search, including strategies for chunking and indexing, embeddings, vector and lexical search, reranking, and evaluating retrieval quality.

• Expertise in creating developer tools for production agents, potentially including experience with multi-agent orchestration, agent and tool registries, and shared infrastructure for tracing and evaluation.

• Proven ability to manage platform initiatives from design to adoption across retrieval and indexing, LLM provider integrations, and agent orchestration.

• Exhibit technical leadership on impactful projects and complex technical challenges.

• Solid track record of delivering intricate projects involving multiple stakeholders, dependencies, or phases.

• Strong backend engineering fundamentals.

• Proficient in Ruby on Rails, Python, JavaScript, or Go.

• Experience with SQL and RDBMS; PostgreSQL is preferred.

• Hands-on familiarity with the AI application lifecycle, including experiment design, dataset curation, offline evaluation, error analysis, deployment, and online evaluation.

• Capability to navigate ambiguity and foster clarity.

• Excellent cross-functional communication abilities.

• An ownership mindset coupled with a bias for action.

• A genuine curiosity, commitment to continuous learning, and an agent-focused approach.

• Nice to have: pgvector or similar vector indexes and query performance at scale.

• Nice to have: content ingestion from external systems and lifecycle management.

• Nice to have: conversation or call transcript processing, including PII and consent considerations.

• Nice to have: evaluation suites for thoroughness and faithfulness.

• Nice to have: expertise in voice or real-time conversational agents.

• Nice to have: experience in enterprise SaaS, B2B marketing or sales technology, or Salesforce/CRM integrations.

• Nice to have: startup experience with a focus on rapid deployment.

🏝️ Benefits

• Time off programs.

• Medical insurance.

• Dental insurance.

• Vision insurance.

• Mental health support.

• Paid parental leave.

• Life insurance.

• Disability insurance.

• 401(k).

• Employee stock purchasing program.

• Potential incentive compensation.

• Possible equity opportunities.

• Reasonable accommodations during the application or recruitment process.