Senior Storage Software Engineer – DGX Cloud

This is a fully remote position, open to applicants in California.

📋 Description

• Contribute code to open-source parallel and distributed file systems, as well as distributed object storage.

• Engage with upstream communities and maintainers by upstreaming fixes and features.

• Serve as a hands-on storage software lead, responsible for writing and reviewing production code.

• Diagnose bugs by analyzing kernel, NFS, NVMe-oF, or SPDK source code.

• Make critical technical decisions on storage deliveries based on measurable targets.

• Triage, troubleshoot, and root-cause complex storage issues across extensive GPU clusters.

• Investigate I/O and metadata performance, as well as issues related to data corruption and recovery.

• Validate the architecture, capabilities, performance, and durability of storage solutions.

• Conduct scale tests, benchmarks, and recovery drills to ensure system reliability.

• Qualify new builds according to defined performance and durability metrics.

• Define and recommend best practices for configuration, tuning, and operations of high-performance file systems on GPU infrastructure.

• Assist operators and internal customers in applying storage guidelines effectively.

• Collaborate with teams in training, inference, accelerated computing, SRE, operations, networking, and security.

• Work alongside cloud providers, neocloud operators, and storage vendors to develop common architecture.

• Utilize modern AI coding and agentic tools to enhance building, debugging, validation, and operational processes.

⛳️ Requirements

• BS, MS, or PhD in Computer Science, Electrical Engineering, or a related field — or equivalent experience.

• More than 12 years of direct experience in storage software engineering.

• Extensive experience with high-performance parallel or distributed file systems managing multi-petabyte scale.

• Contributions to open-source projects that involve distributed or parallel file systems.

• Hands-on experience in writing and reviewing production code, examining file system, kernel, NVMe-oF, or SPDK source, and conducting scale tests or recovery drills.

• Proven ability to diagnose and resolve storage challenges in large GPU or HPC clusters, including I/O and metadata performance analysis.

• Strong expertise in at least one systems programming language: C, C++, Rust, or Go.

• Proficiency in Python programming.

• Comfortable working with Linux kernel storage and networking stacks, including block layer, RDMA/RoCE/InfiniBand, NVMe, page cache, VFS, and multipath configurations.

• Solid understanding of object storage concepts, including S3/Swift-class implementations.

• Strong grasp of block storage technologies, including NVMe-oF and iSCSI.

• Excellent written and verbal communication skills.

• Ability to operate in a 24/7 production environment.

• Security-first mindset.

• Maintainers or contributors to widely used public projects.

• Experience in crafting or operating storage solutions for AI training or inference at a large GPU scale.

• Background in kernel and file system development, metadata scalability, data placement, failure recovery, or equivalent experience.

• Experience with Kubernetes and CSI driver development for storage solutions.

• Hands-on experience with SPDK, libfabric, or FUSE performance optimization techniques.

🏝️ Benefits

• Equity.

• Benefits.