Senior AI Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and implement production-level AI agents and comprehensive workflows that address genuine business challenges within ACQ Vantage.

• Integrate large language models (LLMs) with internal systems, APIs, and data sources, ensuring their reliability, performance, and clean abstractions.

• Collaborate with product and engineering teams to prioritize, launch, and iterate on AI features swiftly.

• Take ownership of and enhance RAG pipelines across various Pinecone namespaces, which includes chunking strategies, selection of embedding models, hybrid retrieval methods, and reranking processes.

• Construct and uphold an evaluation framework, encompassing golden datasets, automated quality assessments, retrieval metrics, latency benchmarks, and regression detection.

• Optimize model routing and tiering to enhance unit economics while ensuring high output quality.

• Equip the AI layer for observability, tracking metrics such as cost-per-request, token usage, quality indicators, and anomaly detection.

⛳️ Requirements

• Over 7 years of experience in deploying production software systems, including distributed backends, APIs, deployment pipelines, and monitoring.

• At least 2 years of experience in developing production RAG systems using vector databases like Pinecone, Qdrant, FAISS, or Weaviate, with a focus on embedding strategies, index management, and retrieval optimization.

• Experience in building and deploying AI agents or multi-step LLM workflows in a production environment, including tool utilization, orchestration, and system integrations.

• Contributed to or developed an evaluation framework for an LLM-centric product, focusing on retrieval quality assessment, regression detection, and model-switching decisions based on data analysis.

• Achieved reductions in LLM API costs in production through model routing, caching, token management, or architectural enhancements.

• Worked with various LLM providers such as OpenAI or Anthropic, possessing a clear understanding of trade-offs in prompt behavior, token economics, and potential failure modes.

• Proficient in both TypeScript and Python, as our technology stack utilizes both languages.

🏝️ Benefits

• Flexible Unlimited Paid Time Off and Company-wide Holidays.

• Employer-sponsored Medical, Dental, & Vision plans.

• Annual Employer HSA contribution of $1,950.

• FSA options available, including dependent care.

• Access to an employee assistance program and mental health resources.

• 401(k) employer match program, applicable for both Traditional and Roth accounts.

• Annual wellness reimbursement of $1,200 through JOON, supporting health, family care, pet care, fitness, and more!

• For local or visiting team members, enjoy access to a state-of-the-art gym at our headquarters in Las Vegas.