Lead DevOps Engineer – AI, GPU

This is a fully remote position, open to applicants in India.

📋 Description

• Develop and manage CI/CD/CV pipelines for AI services, agents, and runtime components, encompassing automated build processes, testing, continuous verification, release management, and rollback procedures.

• Take ownership of governance and discovery for AI assets, which includes service registries, experiment tracking, model metadata, upstream registries, CVE/SBOM scanning, lineage contracts, and license management.

• Manage resources and costs, including quotas, rate limits, cost attribution, and utilization metrics.

• Oversee the observability stack that includes metrics, logs, traces, dashboards, LLM debugging/evaluation, and SLA/alert notifications.

• Lead the OpenTelemetry collection layer, which involves multi-tenant receivers, exporters, queues, GPU telemetry, batching, and dynamic filtering.

• Automate GitOps-based delivery and continuous verification, ensuring quality, integrity, and adherence to cost gates.

• Utilize telemetry, evaluation, and cost signals to inform deployment decisions, facilitate progressive rollouts, and enable automated rollback.

• Guarantee that costs and telemetry are identity-stamped and attributable for each tenant throughout the entire process.

⛳️ Requirements

• Extensive DevOps expertise in designing CI/CD/CV pipelines, implementing GitOps, and executing continuous verification, along with progressive and automated release and rollback strategies for production workloads.

• In-depth experience in operating model-serving and inference systems (Ray, vLLM/Triton/NIM) on GPUs at a production scale.

• Strong skills in observability, including metrics, logs, traces, and OpenTelemetry.

• Capability to attribute, bound, and optimize costs associated with AI consumption per tenant and workload.

• Knowledge of model/artifact governance, registries, CVE scanning, and license/lineage tracking.

• Experience working across cloud, on-premises, edge, and air-gapped environments.

• Bachelor’s or Master’s degree in Computer Science or a related technical field.

• Minimum of 8 years in DevOps, MLOps, platform, or observability engineering, with direct production ownership of AI or high-throughput services.

• Practical experience with ArgoCD, Helm, GitHub Actions/GitLab CI, or similar tools.

• Proficient in using Ray Serve, vLLM, Triton, or NIM on GPU infrastructure.

• Familiarity with Prometheus, Grafana, Loki, Tempo/Jaeger, and OpenTelemetry.

• Experience with API gateways and request routing technologies such as Envoy, including support for streaming responses.

• Knowledge of OpenCost, Kubecost, or similar solutions, including quota and rate-limit enforcement.

• Familiar with Harbor, MLflow, Trivy/SBOM, and supply-chain scanning methodologies.

• Experience managing AI or service infrastructure under compliance, security, or regulatory constraints.

• Ability to delineate ownership boundaries and establish consumption contracts with platform, trust, and data teams.

🏝️ Benefits

• Comprehensive health, dental, and vision insurance.

• Generous paid time off and holidays.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• Collaborative and inclusive company culture.