Senior Technical Program Manager – AI Tooling & Systems

This is a fully remote position, open to applicants in United States.

📋 Description

• Take charge of the complete delivery of AI infrastructure initiatives—from establishing model training pipelines and tracking experiments to managing inference serving and production oversight.

• Establish the technical architecture, integration patterns, and implementation strategies for new machine learning systems and tools (such as vector databases, model servers, evaluation frameworks, and prompt engineering platforms).

• Act as a vital link among ML research, ML engineering, product, and data teams to ensure alignment on ML system specifications, capability roadmaps, and deployment schedules.

• Lead efforts to optimize costs and reduce latency for real-time inference workloads at scale.

• Develop streamlined internal tools and processes to expedite ML iteration cycles (including experiment tracking, model versioning, and A/B testing frameworks).

• Detect and address technical constraints in training pipelines, serving frameworks, and model evaluation processes.

• Collaborate closely with ML practitioners to convert research advancements into scalable, observable systems.

⛳️ Requirements

• Over 5 years of experience in program management or technical leadership within ML infrastructure, ML platforms, or AI tools (or a comparable background).

• Strong technical expertise in ML systems—preferably with hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer.

• Proven experience in coordinating cross-functional ML initiatives (e.g., from model training to evaluation, serving, and monitoring).

• Demonstrated ability to convert ML/research requirements into durable, scalable infrastructure.

• Comfortable navigating ambiguity and aiding teams in managing intricate technical trade-offs (such as accuracy versus latency versus cost).

• Exceptional communication skills with both technical and non-technical audiences.

• Experience in high-growth or startup environments is a plus.

• It Would Be Great If You Had

• Practical experience with model serving frameworks (like vLLM, TensorRT, TorchServe, or similar).

• Experience in optimizing inference for LLM or speech/audio models (including quantization, distillation, KV-cache optimization, and batching strategies).

• Familiarity with ML experiment tracking and versioning tools (such as MLflow, Weights & Biases, DVC, or similar).

• Background knowledge in feature stores, vector databases, or real-time ML systems.

• Understanding of cost optimization strategies for GPU/ML workloads on both cloud and on-premise infrastructures.

• Experience with multi-region model serving or edge deployment.

• Familiarity with relevant frameworks (like PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (such as AWS SageMaker, GCP Vertex AI, Azure ML).

🏝️ Benefits

• Provides Equity

• Includes Bonus

• 10% Annual Bonus