Distributed AI Scheduling Engineer

This is a fully remote position, open to applicants in Ukraine.

📋 Description

• Construct Ray-based distributed task graphs for tasks such as inference, training, RAG, vector search, document retrieval, video, and compound workloads.

• Design deployment patterns for Ray head and worker on Kubernetes.

• Develop workload drivers tailored to specific hardware for GPU inference, vector search, storage retrieval, video acceleration, and edge inference.

• Implement logic for model locality and cache awareness to ensure workloads operate close to the appropriate model or data.

• Guarantee the integration of scheduling decisions, capacity state, SLA rules, billing events, and observability traces.

• Execute comprehensive end-to-end workload tests for compound RAG and multi-plane AI tasks.

⛳️ Requirements

• Proven experience in Python distributed systems.

• Practical experience with Ray, especially in production environments, KubeRay, or orchestration of distributed AI workloads.

• Familiarity with AI inference, training, RAG pipelines, model serving, model locality, and scheduling that is aware of accelerators.

• Experience with Kubernetes for deploying distributed computing systems.

• Proficiency in debugging across Python, Kubernetes, GPUs, network movement, and application latency.

• Nice to Have: Experience with vLLM, TensorRT-LLM, KubeRay, vector search, RDMA storage, GPU scheduling, video pipelines, edge inference, or large-scale model-serving platforms.

🏝️ Benefits

• Competitive Compensation: Your salary is aligned with your qualifications, experience, and performance.

• Exceptional Stability: Enjoy job security along with numerous opportunities for career advancement.

• Professional Development: Gain access to educational programs and certifications to enhance your skills.

• Health & Wellness: Comprehensive support for your fitness needs.

• Flexible Working Environment: Take advantage of a fully remote work setup, flexible schedules, and relocation assistance if necessary.

• Performance Recognition: Benefit from regular bonuses, annual salary reviews, and acknowledgment of your accomplishments.

• Advanced Equipment Options: Select the workstation setup that best suits your needs, whether it be a desktop or laptop.

• Team Building & Community: Engage in regular events that promote collaboration, camaraderie, and innovation.