Senior Software Engineer – AI/HPC Middleware

This is a fully remote position, open to applicants in Costa Rica.

📋 Description

• Implement and enhance functionalities for HPC middleware (HPI and SHMEM) and AI middleware, which includes CCL stacks like NCCL/RCCL.

• Integrate middleware features with transport and provider layers, such as libfabric/OFI, UCX, and relevant verbs-like semantics.

• Prepare and submit upstream contributions across MPI/SHMEM projects, CCL ecosystems, and associated components.

• Collaborate with kernel/driver and switch teams to achieve end-to-end performance that aligns with the product roadmap.

• Analyze performance traces and replicate customer issues.

• Convert customer and field insights into actionable fixes.

• Contribute to the design, development, and upstream enablement of AI/HPC communication middleware.

• Drive performance, correctness, and deployability across genuine customer workloads.

⛳️ Requirements

• Over 6 years of experience in systems programming using C/C++ on Linux.

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience.

• Practical experience with HPC middleware internals, including Open MPI, MPICH, MVAPICH, and/or SHMEM.

• Experience in optimizing AI collective communications, such as NCCL/RCCL, CUDA/ROCm, and/or other CCL stacks.

• Capability to deliver low-latency/high-throughput communication pathways.

• Skill in diagnosing performance challenges through profiling and tracing tools.

• Working knowledge of OFI/libfabric, UCX, and verbs-style concepts.

• Strong background in RDMA concepts and performance optimization.

• Experience contributing to open-source projects.

• Previous experience in developing or maintaining libfabric providers is preferred.

• Familiarity with Ultra Ethernet (UEC/UET) specifications is preferred.

• Experience with RoCEv2, congestion control, and/or Ethernet-based RDMA implementations is preferred.

• Experience with cluster-scale benchmarking, profiling, and optimization is preferred.

• Background with HPC fabrics such as Omni-Path is preferred.

🏝️ Benefits

• Performance incentives.

• Equity participation.

• Additional benefits based on geographic location.

• Paid holidays.

• Flexible work arrangements.