Senior LLMOps Engineer, Development

This is a fully remote position, open to applicants in United States.

📋 Description

• Create and establish automated CI/CD deployment pipelines specifically for generative AI applications.

• Safely transition prompt, LangGraph state-machine, and RAG retrieval updates across Development, Testing, and Production environments.

• Deploy and oversee continuous AI evaluation infrastructure.

• Incorporate precision, recall, and toxicity checks into deployment gates.

• Instrument AI applications and develop dashboards to monitor token consumption, latency, reasoning traces, and API failure rates.

• Execute version control and maintain auditable registries for prompts and model configurations.

• Integrate guardrails and content filtering to prevent prompt injection, PII leakage, and irrelevant responses.

• Observe AI financial usage, establish token-spike alerts, and enhance embedding and retrieval strategies.

• Collaborate with AI Engineers to industrialize generative AI applications and facilitate enterprise AI operations.

⛳️ Requirements

• Bachelor’s degree in computer science, information technology, engineering, or a related practical experience.

• Databricks Certified: Machine Learning Professional.

• Microsoft Certified: Azure DevOps Engineer Expert (AZ-400).

• DeepLearning.AI: Machine Learning Engineering for Production (MLOps).

• 4+ years of experience in DevOps, MLOps, or Site Reliability Engineering (SRE).

• Relevant hands-on experience managing generative AI deployments in the past 1–2 years.

• Expertise in building CI/CD pipelines using Azure DevOps, GitHub Actions, or GitLab CI.

• Practical experience with LLMOps tools and frameworks such as MLflow, LangSmith, PromptFlow, or Arize.

• Strong skills in Python scripting.

• Experience with containerizing machine learning or API workloads using Docker and Kubernetes.

• Familiarity with frontier-model APIs including OpenAI, Anthropic, and Google Vertex AI.

• Understanding of multi-agent frameworks such as LangChain and LangGraph.

• Knowledge of Azure, AWS, and infrastructure-as-code principles.

🏝️ Benefits

• Competitive compensation and benefits.

• Support for professional development.

• Flexibility to balance personal and professional life.