
Principal Engineer, LLM
Posted 9 hours ago

Posted 9 hours ago
This is a fully remote position, open to applicants in United States.
• Develop and maintain an agent platform that empowers non-technical users to create, launch, and manage voice, chat, and text AI agents.
• Create orchestration for a cohesive personal assistant that shares context across specialized agents.
• Enhance internal workflows connected to borrowers.
• Quickly prototype high-ambiguity, 0-to-1 initiatives across platforms for agents, orchestration, borrower-facing assistants, and internal workflows.
• Collaborate with Product and Design teams to validate non-deterministic user experiences and set architectural frameworks.
• Provide golden datasets, automated evaluation tools, failure-mode maps, latency and cost benchmarks, and architectural designs.
• Work alongside engineering teams from technical exploration through production stages, including making go/no-go decisions, PR reviews, and troubleshooting model performance.
• Design agent transitions, fallback strategies, and reliable non-deterministic user experiences.
• Make informed decisions regarding prompting, guardrails, model routing, fine-tuning, and distillation processes.
• Advocate for agentic coding and verification practices and elevate applied AI capabilities across teams.
• Over 8 years of experience in software engineering.
• Practical experience in deploying non-deterministic, user-facing LLM or AI products to actual users.
• Extensive hands-on experience designing production-level LLM systems that involve structured outputs, tool calling, context engineering, retrieval/memory patterns, and orchestration involving multiple steps or agents.
• Solid understanding of distributed systems and backend fundamentals, including asynchronous orchestration, streaming user experiences, caching, timeouts, fallback paths, rate-limit management, and comprehensive observability/tracing.
• Demonstrated experience in creating golden datasets, automated evaluation harnesses, regression testing gates, and thorough failure-mode analyses.
• Strong judgment in the areas of deterministic logic, prompting, context engineering, guardrails, routing, fine-tuning, and distillation.
• Proficiency in backend and frontend prototyping using languages and frameworks such as Python, TypeScript/React, and modern REST or streaming APIs.
• Proven ability to collaborate effectively with partner teams and leaders in Product, Design, operations, or business sectors.
• Bachelor's degree in Computer Science, Engineering, Mathematics, or equivalent professional experience.
• Availability to work in East or West coast time zones.
• Preferred: experience in startups or 0-to-1 incubation, familiarity with agent platforms/shared abstractions, production fine-tuning/distillation, AI-native engineering practices, and experience in regulated or high-trust consumer domains.
• Competitive salary with opportunities for bonuses and annual equity grants that vest quarterly.
• Retirement benefits including a 401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually.
• Employee Stock Purchase Plan (ESPP) offering discounted stock purchase options for eligible employees (US only).
• Comprehensive medical, dental, vision, and wellness coverage/resources.
• Supplemental health coverage for employees in Canada.
• Contributions to Health Savings Accounts for eligible plans (US only).
• Life insurance and disability coverage.
• Paid time off, sick leave, and company holidays.
• Paid family and parental leave.
• Family-focused benefits for fertility, parenthood, and caregiving.
• Employee Assistance Program providing mental health support and life-centered resources.
• Financial planning tools and concierge services (US only).
• Annual wellness allowance.
• Annual productivity allowance for relevant tools and resources.
• Team events, company-wide updates, and employee resource groups (ERGs).
• Catered lunches and fully stocked micro-kitchens at offices in the Bay Area, Austin, Columbus, and New York City.
datavent.io
CVS Health Careers
CVS Health
Remofirst
Get handpicked remote jobs straight to your inbox weekly.