Senior AI Storage Infrastructure Engineer

This is a fully remote position, open to applicants in California, +1 more state.

📋 Description

• Design, implement, and sustain CSI drivers for high-performance parallel file systems such as Weka, Lustre, DAOS, and VAST.

• Architect and execute GPUDirect Storage integrations for DMA operations between NVMe drives and GPU memory.

• Develop and oversee local NVMe caching strategies for model weights and datasets during distributed training.

• Enhance IOPS, throughput, and latency across the containerized storage architecture.

• Collaborate with the GPU Systems & Fabric team to refine storage for RDMA and high-speed interconnects.

• Establish automated monitoring and alerting systems for storage performance, addressing I/O contention or hardware degradation.

• Define storage policies, manage quotas, and implement Kubernetes multi-tenancy isolation strategies.

• Mentor junior engineers and spearhead architectural design reviews.

⛳️ Requirements

• Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or a related field.

• Over 5 years of experience in distributed storage systems and high-performance file systems.

• Profound understanding of POSIX compliance and file I/O semantics.

• Extensive knowledge of the Kubernetes CSI paradigm, including the development or extension of volume plugins and storage operators.

• Strong practical experience with block/file I/O at the Linux OS level and kernel-level performance optimization.

• Familiarity with RDMA, InfiniBand, and RoCE, including their interactions with storage subsystems.

• Proven experience in operating, troubleshooting, and scaling large-scale storage environments in production or HPC scenarios.

• Proficient with Terraform, Ansible, and CI/CD pipelines.

• Exceptional technical communication skills and the ability to influence cross-functional architectural decisions.

• Experience in fast-paced, high-growth engineering environments is highly preferred.

🏝️ Benefits

• Equal employment opportunities and non-discrimination protections in accordance with country, state, and local laws.