AI Infrastructure Engineer – Tooling

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and take ownership of tooling and services that transition AI agents from development to production within customer cloud environments.

• Create and maintain internal CLIs, SDKs, and services using Python or Go.

• Deploy, scale, and optimize LLM inference utilizing vLLM and similar serving architectures.

• Construct infrastructure for multi-agent systems, which encompasses agent runtimes, tool and connector execution, sandboxing, state management, and secure agent-version rollouts.

• Oversee MLOps workflows that include model and prompt versioning, evaluation pipelines, artifact registries, and the promotion process from staging to production.

• Develop Kubernetes operators and custom controllers that manage AI workloads effectively.

• Design, implement, and maintain production-grade AI agent infrastructure leveraging Kubernetes and Terraform.

• Ensure portable infrastructure across Azure, Oracle Cloud, GCP, and AWS, maintaining primary ownership of at least one cloud variant.

• Enhance observability for agent and LLM calls, including token and GPU utilization, latency, error rates, and costs.

• Fortify security and reliability for workloads in customer environments, adhering to strict compliance requirements.

• Design and manage CI/CD for infrastructure, services, and model/agent releases using GitOps and pipeline-as-code methodologies.

• Develop self-service tools for environment provisioning, secret rotation, dependency updates, compliance checks, and release management.

• Collaborate with AI, product, and platform engineers to facilitate delivery and remove obstacles.

⛳️ Requirements

• A minimum of 5 years of experience in infrastructure, platform, or backend engineering (7+ years for senior positions).

• Strong proficiency in software engineering using Python or Go, with a track record of building and deploying tools, CLIs, or services utilized by fellow engineers.

• Practical experience managing LLM workloads in production, including vLLM or a similar serving stack (e.g., TGI, Triton, SGLang, Ray Serve).

• Experience in building infrastructure for AI agents or LLM applications, covering orchestration, tool invocation, evaluation, or agent runtimes.

• Hands-on experience with MLOps, including model/prompt versioning, evaluation and release pipelines, and ML observability.

• Strong hands-on experience with Kubernetes, specifically in writing or extending operators/controllers using Kubebuilder, Operator SDK, controller-runtime, or Kopf.

• Solid experience with Terraform and at least one of the following cloud platforms: Azure, Oracle Cloud, GCP, or AWS.

• Proven expertise in designing and managing CI/CD systems, GitOps-driven deployments, and release automation.

• Solid understanding of networking fundamentals.

• Demonstrated experience in deploying and operating production systems.

• Familiarity with GPU infrastructure, such as NVIDIA GPU Operator, MIG, KServe, or Karpenter-style autoscaling for inference, is advantageous.

• Knowledge of LangGraph, Model Context Protocol, OpenAI Agents SDK, or similar technologies is a plus.

• Familiarity with OpenTelemetry, Langfuse, or Arize is also beneficial.

• Experience with FluxCD, ArgoCD, or other GitOps controllers is desirable.

• Cloud security and DevSecOps experience is a plus.

• Extensive networking expertise is a benefit.

• Contributions to open-source projects in the AI infrastructure or Kubernetes ecosystem are valued.

🏝️ Benefits

• Equity.

• Work on state-of-the-art AI infrastructure with significant enterprise impact.

• Take ownership of foundational systems at an early-stage, high-growth company.

• Collaborate with a talented, execution-focused founding team.

• Opportunity to influence the deployment of AI agents across enterprise platforms.