Principal Software Engineer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Take ownership of the architecture for Enterprise AI, encompassing the AI gateway, control plane, model serving, identity, metering, and observability.

• Establish high technical standards for a small, senior, remote-first team.

• Lead the strategy for the gateway, focusing on data plane selection and policy-based routing, including per-request metering and audit records.

• Provide direction for LLM serving on Kubernetes, addressing GPU scheduling and sharing, autoscaling, multi-tenancy, disaggregated serving, and cost per token.

• Define enterprise deployment practices, which include Helm installations and upgrades, air-gapped networks, preflight checks, and supply-chain security.

• Conduct design reviews, make decisions between building and buying solutions, and document these decisions as Architectural Decision Records (ADRs).

• Engage in hands-on work in critical-path areas.

• Guide and mentor senior engineers, particularly in AI-assisted development.

• Collaborate with design partners and customers.

• Represent Mirantis within the Kubernetes and ML-infrastructure communities.

⛳️ Requirements

• Over 10 years of experience in infrastructure or distributed systems, with several years in a technical lead role.

• Proven track record of delivering multi-service systems that enterprises operate in production, including software that customers install independently.

• Expertise in Kubernetes, including operators, CRDs, scheduling, Helm, and debugging at scale.

• Production experience with GPU workloads, LLM inference, or AI gateways.

• Proficient in Go.

• Strong programming skills in Python.

• Solid knowledge of CI/CD and infrastructure as code practices.

• Proficient in utilizing AI-assisted development tools.

• Ability to excel with autonomy in a distributed team environment.

• Nice to have: experience in inference performance work, including quantization, batching, KV-cache management, and disaggregated prefill and decode.

• Nice to have: experience with AI gateway or LLM routing, including failover, rate limiting, token metering, and policy enforcement.

• Nice to have: experience in launching a new product to its initial enterprise customers.

🏝️ Benefits

• Competitive compensation package.

• Comprehensive benefits plan.

• Flexible remote work arrangement.