Senior Staff Engineer, Lustre

This is a fully remote position, open to applicants in North Carolina, +1 more state.

📋 Description

• Design, develop, and troubleshoot LustreFS features, enhancements, and fixes across llite, MDS/MDT, OSS/OST, LDLM, and LNet.

• Analyze customer and scale-related defects, conduct root-cause investigations, and implement high-quality solutions.

• Contribute to performance optimization, failure analysis, and reliability enhancements for large-scale Lustre deployments.

• Engage in code reviews, design evaluations, and subsystem discussions.

• Collaborate with QE and support teams to replicate issues, enhance diagnostic data quality, and improve coverage for high-risk failure scenarios.

• Document subsystem behaviors, debugging methods, known failure patterns, and operational best practices.

• Utilize AI-assisted tools to expedite issue triage, log summarization, code comprehension, and the application of reusable lessons learned.

• Investigate customer-facing and scale-related challenges across client, metadata, object storage, LNet, and transport layers.

• Design and implement enhancements for reliability, observability, and serviceability.

• Review and validate fixes through targeted testing, failure injection, log analysis, and performance characterization.

• Enhance documentation, code review standards, and the sharing of subsystem knowledge.

⛳️ Requirements

• Over 10 years of experience in systems software, distributed systems, storage, Linux kernel, or filesystem engineering.

• Extensive experience in LustreFS development, support, or performance engineering, with expertise in at least one major subsystem.

• Proficient in C programming and debugging within Linux systems.

• Familiarity with Linux kernel internals, filesystem semantics, networking, and performance analysis.

• Experience with LNet and/or high-performance transports such as RDMA, InfiniBand, RoCE, or TCP-based storage networking.

• Ability to troubleshoot and resolve issues across client, server, network, and backend storage layers.

• Strong collaborative skills, with the ability to work across different functions in a dynamic engineering environment.

• Preferred experience in HPC, AI infrastructure, or large-scale parallel storage environments.

• Familiarity with metadata-heavy and throughput-heavy workload characterization and tuning is preferred.

• Knowledge of ZFS, ldiskfs, NVMe-backed storage, and observability/performance tools is preferred.

• Experience in creating test plans, reproducer frameworks, runbooks, or diagnostic automation is preferred.

• Comfort in using AI tools for debugging, code reviews, triage, documentation, and early-stage design ideation is preferred.

• Experience mentoring junior engineers or leading focused technical projects within a subsystem is preferred.

🏝️ Benefits

• No specific benefits, perks, or compensation extras are mentioned in the job posting.

• Application limit: up to 3 positions within a 60-day period.

• Candidates are advised to wait 90 days before reapplying for the same position if it does not align with their qualifications.