
Senior Software Engineer
Posted Nov 11

Posted Nov 11
This is a fully remote position, open to applicants in North Carolina.
• Design and develop infrastructure for LLM serving on Kubernetes, encompassing deployment, GPU scheduling, scaling, and management of the model lifecycle.
• Package the platform for enterprise use with Helm-based installations and upgrades, accommodating restricted or offline networks.
• Integrate the serving layer with the platform’s API gateway, identity management, and metering services.
• Create observability features for operating GPU inference in production, including serving metrics and GPU telemetry.
• Contribute to a multi-service codebase.
• Assist in defining engineering direction through design documents and code reviews.
• Over 5 years of experience in software engineering, focusing on infrastructure, platforms, or distributed systems.
• Extensive hands-on experience with Kubernetes for building and managing production workloads and Helm charts.
• Familiarity with GPU workloads or LLM inference, or a strong background in adjacent systems with a proven ability to learn quickly.
• Proficient in Go programming.
• Strong skills in CI/CD and infrastructure-as-code methodologies.
• Comfortable using AI-assisted development tools, such as Claude Code and OpenAI Codex, as part of the daily engineering process.
• Ability to work autonomously in a small, remote-first team with a strong written culture.
• Experience in inference performance, enterprise deployment, UI development, and open-source contributions is a plus.
• Competitive compensation package.
• Comprehensive benefits plan.
• Opportunities for open-source innovation.
• Collaboration with skilled colleagues.
• Environment conducive to continuous growth.
Zen Educate
Weekday (YC W21)
Toast
Milliman
Get handpicked remote jobs straight to your inbox weekly.