Principal AI Ops Engineer

This is a fully remote position, open to applicants in New York.

📋 Description

• Develop and maintain comprehensive deployment pipelines for AI-driven applications, which encompass artifact builds, environment promotions, rollbacks, and observability features.

• Establish and manage the runtime and lifecycle infrastructure for production agents, covering deployment, versioning, monitoring, rate-limiting, and decommissioning.

• Oversee the organization's provisioning, rotation, scoping, and metering of access to model provider APIs (including Anthropic, OpenAI, and others).

• Integrate evaluations into the CI/CD pipeline to ensure that no agent or LLM-powered service is deployed without meeting a specified evaluation standard.

• Implement self-hosted inference for workloads that do not align well with managed APIs.

• Design and create a shared developer toolkit that is utilized by all AI-powered services.

• Collaborate with the finance team to enhance cost visibility, which includes token accounting and assigning costs to specific features.

• Produce documentation, runbooks, and maintain clear user interfaces.

• Engage in code reviews to foster collaboration and uphold best practices.

⛳️ Requirements

• Bachelor’s degree (or higher, such as a Master’s or Ph.D.) in Computer Science or a related technical discipline involving programming, or a comparable level of technical experience.

• Over 6 years of experience in platform, infrastructure, or DevOps engineering, with a minimum of 2 years focused on developing production infrastructure for AI/ML or LLM-driven systems.

• Extensive hands-on experience in designing and managing CI/CD pipelines for high-velocity engineering teams, which includes artifact management, environment promotion, and progressive rollout strategies.

• Strong background in AWS, with proficiency in IAM, networking, and container orchestration layers.

• A proven history of building developer platforms or internal tools that are willingly adopted by other engineering teams, rather than enforced.

• Practical experience with LLM-powered applications, including prompt management, model routing, retries, tracing, and the operational challenges of running agents or chains in a production environment.

• Proficient coding skills in Python or TypeScript, preferably both.

• Comfortable working in a polyglot programming environment.

• Able to manage discussions regarding cost and reliability with both engineering leadership and finance partners.

• Strong written communication skills with a focus on documentation, runbooks, and clear interfaces.

• Demonstrated analytical thinking and problem-solving capabilities.

• Excellent verbal and written communication abilities.

🏝️ Benefits

• RxSense is committed to fostering a diverse workforce, believing that it leads to a more talented and effective team. We are an Equal Opportunity and Affirmative Action employer, ensuring our hiring process is free from discrimination. All qualified applicants are considered for employment regardless of race, color, religion, sex, gender, sexual orientation, gender identity, ancestry, age, or national origin.

• Qualified applicants will also not face discrimination based on disability or veteran status.

• We recognize the value of collaboration, creativity, and the sense of community that a diverse workforce contributes.