Kong AI Gateway Engineer

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Act as a technical lead concentrating on Kong AI Gateway.

• Design, architect, build, test, and support contemporary digital products.

• Oversee applications throughout their production lifecycle and drive overall success against key performance indicators.

• Create and implement enterprise-level AI Gateway solutions utilizing Kong AI Gateway.

• Configure AI-specific policies such as prompt logging, content filtering, token management, and guardrails.

• Facilitate secure access to various LLM providers through a unified gateway architecture.

• Implement AI observability, token analytics, usage monitoring, and cost governance.

• Develop routing strategies across multiple AI models based on factors like latency, cost, and availability.

• Integrate RAG-based AI services via secure API management frameworks.

• Ensure adherence to enterprise AI governance, security, and regulatory standards.

• Establish AI platform standards, reusable AI APIs, and best practices for GenAI adoption.

• Support AI agent and MCP-based integrations within enterprise ecosystems.

• Promote reliability, scalability, and operational excellence of the AI platform.

• Configure, manage, and support Kong Gateway environments.

• Develop and manage AI Gateway proxies for GenAI/LLM services.

• Enforce API security, authentication, throttling, and governance policies.

• Integrate AI platforms such as Azure OpenAI and OpenAI through secure API gateways.

• Deploy and manage Kong in Kubernetes-based environments.

• Monitor, troubleshoot, and enhance gateway performance and availability.

• Automate deployments and operational processes using DevOps tools.

⛳️ Requirements

• Candidates should be able to work during Central Time or Eastern Time hours.

• Over 8 years of experience in API Management, Middleware, or Platform Engineering.

• At least 3 to 5 years of hands-on experience with Kong Gateway Administration.

• Preferred experience with cloud-native and microservices architectures.

• Familiarity with Kong Gateway (OSS/Enterprise), Kong Konnect.

• Expertise in API Management & API Lifecycle Management.

• Experience in AI Gateway / LLM Proxy Development.

• Knowledge of OpenAI / Azure OpenAI Integration.

• Proficiency in REST APIs, GraphQL.

• Understanding of OAuth2, JWT, OIDC, API Security.

• Experience with Kubernetes, Docker, Helm.

• Familiarity with CI/CD tools (Jenkins, GitHub Actions, Azure DevOps).

• Monitoring experience with Prometheus, Grafana, ELK/Splunk.

• Proficiency in cloud platforms such as Azure, AWS, or GCP.

• Knowledge of Kong AI Gateway Configuration & Administration.

• Experience with LLM Traffic Routing, Load Balancing, and Failover Management.

• Familiarity with Prompt Management, Prompt Security, and Guardrails.

• Understanding of AI Security, Responsible AI, and Model Governance.

• Experience in Token Usage Monitoring, Rate Limiting, and Cost Optimization.

• Knowledge of Multi-LLM Integration (OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Mistral, Llama).

• Familiarity with Retrieval-Augmented Generation (RAG) Architecture Concepts.

• Understanding of Vector Databases.

• Experience in LLMOps, AI Observability, and Model Performance Monitoring.

• Knowledge of AI API Governance, Compliance, and Data Protection.

• Familiarity with Prompt Caching, Semantic Caching, and Response Optimization.

• Experience in Model Gateway Policy Development and AI Consumption Controls.

• Understanding of MCP (Model Context Protocol) and AI Agent Integration Concepts.

• Experience in GenAI Platform Engineering and Enterprise AI Enablement.

• Candidates selected for this position will be required to undergo a background investigation.

• Allstate typically does not sponsor individuals for employment-based visas for this role.

🏝️ Benefits

• Comprehensive technology setup, which includes a laptop, monitors, headset, keyboard, and mouse.

• Monthly connectivity reimbursement for eligible employees working from home.