Engineering Manager, Provider Ecosystem

This is a fully remote position, open to applicants in United States.

📋 Description

• Lead and develop the Provider Ecosystem engineering team.

• Recruit engineers, establish the roadmap, conduct code assessments, manage incidents, and advocate for the team.

• Oversee provider adapters, endpoint lifecycle, and routing processes.

• Act as the technical liaison to key providers.

• Execute day-zero and stealth launches.

• Collaborate with GTM, BD, and legal teams to onboard providers.

• Enhance capabilities in image, video, audio, embeddings, real-time, and batch inference.

• Manage evaluations and ensure inference quality.

• Utilize benchmark and production telemetry data to inform routing decisions and offer feedback to providers.

• Report directly to the VPE.

⛳️ Requirements

• Minimum of 4 years of engineering management experience.

• Proven experience in building or scaling a team during periods of rapid growth.

• Strong background in distributed systems, API design, and LLM inference.

• Knowledge of streaming, tool calling, prompt caching, quantization, and trade-offs related to throughput and latency.

• Familiarity with variations in provider APIs and managing millions of requests across diverse endpoints.

• Proficient in hands-on coding and utilizing AI-assisted development tools.

• Ability to engage with external model labs and engineering teams.

• Experience in recruiting and retaining top engineering talent.

• Demonstrated high agency and effective communication skills.

• A pragmatic approach to lean engineering organizations.

• Nice to have: experience at an inference provider, model lab, GPU cloud, or familiarity with multiple LLM APIs.

• Nice to have: skills in TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, or Vercel.

• Nice to have: knowledge in evaluations, benchmarking, routing, or load-balancing systems.

• Nice to have: experience with multimodal APIs or real-time and batch inference.

• Nice to have: expertise in vendor security and compliance reviews, including NDA, DPA, and SOC 2.

• Nice to have: experience with OpenRouter or AI infrastructure/developer tooling side projects.

🏝️ Benefits

• Equity.

• Remote work.