Senior LLM Ops Engineer

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Design and manage LLM orchestration and runtime systems that facilitate dependable, low-latency AI workflows.

• Develop and sustain evaluation pipelines that assess the quality, regressions, and business impact of LLM-enabled features.

• Implement observability for AI systems, encompassing tracing, metrics, and feedback loops at the prompt, agent, and workflow levels.

• Establish strategies for LLM cost management, which include budgeting, rate limiting, caching, and optimization.

• Collaborate with AI and product engineers to safely and incrementally deploy AI features into production.

• Define and uphold security, privacy, and data-handling guardrails within AI workflows.

• Facilitate experimentation with new models and tools while ensuring stability in production.

• Enhance incident preparedness and response for AI-related failures and performance degradations.

• Influence decisions regarding build-versus-buy for LLM tooling and platforms.

• Mentor engineers and promote best practices for operating AI systems at scale.

⛳️ Requirements

• Over 8 years of experience in software engineering, platform engineering, or DevOps positions.

• Practical experience in running LLM-powered systems in a production environment.

• Familiarity with various LLM providers and orchestration frameworks.

• Strong comprehension of distributed systems, APIs, and cloud-native architectures.

• Experience in designing observability and evaluation systems for intricate workflows.

• Practical understanding of cost and performance optimization within cloud environments.

• Experience with event-driven architectures and asynchronous workflows.

• Proven capability to lead through influence rather than direct authority.

• Highly organized thinker with excellent problem-solving abilities.

• Effective communicator adept at articulating AI operational trade-offs to both technical and non-technical audiences.

• Comfortable collaborating across teams in a dynamic and evolving environment.

• Builder mentality focused on reliability and achieving results.

• Belief that AI enhances engineering fundamentals rather than replaces them.

• Learning-oriented mindset, keeping up with advances in AI tools and practices.

• Pragmatic and composed under pressure, particularly during incidents.

• Customer-focused, understanding the real-world implications of AI behavior.

🏝️ Benefits

• Comprehensive health and wellness benefits.

• Opportunities for professional development and continuous learning.

• Flexible work arrangements to support work-life balance.