GPU Kernel Engineer – CUDA, Triton, Accelerator Performance

This is a fully remote position, open to applicants in Argentina.

📋 Description

• Analyze GPU and accelerator kernel implementations for accuracy

• Validate outputs against established reference implementations

• Assess numerical tolerance levels

• Examine kernel benchmarks to ensure fair comparisons

• Detect performance bottlenecks and identify optimization possibilities

• Evaluate if performance goals are attainable given hardware limitations

• Review kernel translations and transitions across hardware

• Identify issues related to compilation, drivers, memory, shapes, and runtime

• Determine if technical tasks are genuinely challenging or misconfigured

• Offer clear and actionable technical insights

• Implement and troubleshoot kernels

• Enhance the performance of CUDA and Triton kernels

• Translate between different kernel frameworks

• Conduct hardware migrations and operator fusion

• Profile and benchmark system performance

• Confirm numerical accuracy

• Troubleshoot compilation and runtime challenges

• Optimize memory hierarchy and AI workload performance at the kernel level

⛳️ Requirements

• Over 3 years of practical experience in developing, optimizing, or debugging GPU or accelerator kernels

• Significant experience with at least two of the following: CUDA; Triton; NKI / AWS Neuron; Pallas / JAX

• Strong grasp of GPU performance optimization techniques

• Familiarity with kernel profiling tools such as Nsight, NCU, roofline analysis, or framework-specific profilers

• Understanding of memory bandwidth, compute throughput, GPU occupancy, shared memory, register pressure, memory coalescing, and bank conflicts

• Solid comprehension of floating-point numerical accuracy and tolerance levels

• Experience in debugging kernel compilation and runtime problems

• Capability to differentiate between software defects, environmental issues, and genuine optimization challenges

• Experience in developing kernels from technical specifications, translating kernels across frameworks, migrating kernels to different hardware platforms, debugging faulty implementations, optimizing kernel efficiency, and combining multiple operations into optimized kernels

• Exposure to both NVIDIA GPU and custom accelerator ecosystems (preferred)

• Familiarity with AWS Trainium, TPU, JAX, or other accelerators (preferred)

• Experience in compiler engineering (preferred)

• Knowledge of MLIR, XLA, or lowering intermediate representations (preferred)

• Contributions to GPU or ML kernel libraries (preferred)

• Experience with cuBLAS, cuDNN, Triton community kernels, or JAX/XLA custom calls (preferred)

• Background in AI model evaluation, RLHF, or technical benchmark development (preferred)

🏝️ Benefits

• Compensation of $65 per hour

• Part-time, project-based consulting opportunity

• Flexible remote work arrangement