Senior Software Engineer, AI Gateway

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and develop essential systems that drive the AI Gateway, including the model router, MCP gateway, and control plane.

• Create infrastructure for dynamic model selection and routing, automatic failover, cost-based routing, and policy enforcement.

• Manage secure credential storage, session summarization, caching, and rate limiting.

• Develop APIs for OpenAI-compliant interfaces and the Model Context Protocol.

• Establish infrastructure for evaluation, telemetry, replay, and backtesting for agents and LLM workflows.

• Guide architectural choices and mentor engineers as the team expands.

• Optimize performance through caching, session summarization, and rate limiting under production conditions.

• Contribute to roadmap formulation, technical strategy, and cross-functional coordination.

• Create unified abstractions over model and tool providers.

• Enhance credential and identity management for tool access.

• Deliver real-time and historical insights into agent activity with enforceable policies.

• Ensure low latency and high reliability as traffic increases.

⛳️ Requirements

• Over 6 years of backend engineering experience with production-level systems.

• Profound understanding of distributed and highly scalable systems, cloud-native infrastructure, and API design.

• Experience in building secure, high-throughput services such as gateways, proxies, load balancers, or policy engines.

• Proficiency in Go and/or Rust; both languages are preferred.

• Familiarity with AI/ML platforms or model serving infrastructure.

• Strong product-oriented mindset focusing on developer-facing tools.

• Ownership mentality with a focus on shipping, learning, and iterating.

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

• Previous experience with OpenAI, Anthropic, or similar LLM APIs.

• Knowledge of RAG architectures, vector databases, or agent frameworks such as LangChain, AutoGPT, or CrewAI.

• Experience with policy engines such as OPA, observability frameworks like OpenTelemetry, or API gateways.

• Understanding of OAuth2.1, secret management, and cloud IAM systems.

• Experience with Kubernetes, Docker, and microservices architecture.

• Candidates must possess legal authorization to work in the country where they are applying.

• Visa sponsorship is considered on a case-by-case basis based on business requirements.

🏝️ Benefits

• Remote-first approach; work from home, with offices in Seattle and Paris for connection and collaboration.

• Flexible work schedule.

• Generous paid time off (PTO).

• Designated quarterly Whaleness Days.

• Designated end-of-year Whaleness break.

• Home office support.

• Technology stipend equivalent to US$100 net per month.

• Annual learning and development stipend for conferences, courses, certifications, and ongoing education.

• 16 weeks of paid parental leave after six months of employment.

• Equity offered to all full-time employees.

• Medical benefits.

• Retirement benefits.

• Paid holidays.

• Docker swag.