GPU Kernel Engineer

This is a fully remote position, open to applicants in New York.

📋 Description

• Analyze GPU and accelerator kernels for technical accuracy and comprehensiveness

• Examine implementations derived from specifications or reference operators

• Evaluate mathematical behavior, implementation issues, unsupported assumptions, and incomplete solutions

• Oversee kernel development across CUDA, Triton, NKI, and Pallas (JAX)

• Assess framework-specific implementation decisions, execution limitations, translations, and migrations

• Compare various kernel implementations for accuracy and technical excellence

• Evaluate outputs against reference implementations using absolute, relative, and ULP-based tolerances

• Analyze floating-point behavior and precision-related edge cases

• Measure performance using Nsight, Nsight Compute, roofline analysis, and framework-native profilers

• Review benchmarking methodologies, latency, throughput, utilization, memory behavior, and claimed enhancements

• Examine compute- and memory-efficiency optimization strategies, such as tiling, vectorization, parallelization, and workload decomposition

• Investigate registers, shared memory, caches, memory access, data locality, bandwidth utilization, bank conflicts, coalescing, and register pressure

• Diagnose compilation and runtime failures related to drivers, out-of-memory situations, launch configurations, shape or stride mismatches, and autotuning

• Validate task reliability in designated environments and assess debugging techniques

• Evaluate kernel translation, lowering, hardware migration, compiler transformations, and intermediate-representation choices

• Diagnose kernel implementations using outputs, profiler data, runtime behavior, and source code

• Assess operator fusion and the preservation of intended semantics in fused kernels

• Evaluate assigned tasks against structured technical criteria and deliver evidence-based written assessments

• Differentiate valid implementation alternatives from those with technical flaws

⛳️ Requirements

• Over 3 years of hands-on experience in developing, optimizing, or validating GPU or accelerator kernels

• Practical experience with at least two of the following: CUDA, Triton, NKI, or Pallas (JAX)

• Strong grasp of numerical correctness, including absolute, relative, and ULP tolerances

• Experience in selecting and validating suitable reference implementations

• Extensive performance profiling and benchmarking expertise

• Familiarity with Nsight, Nsight Compute, roofline analysis, or similar profiling tools

• Strong understanding of common kernel compilation and runtime failure modes

• Experience in at least three of the following: kernel generation from specification, framework translation or lowering, hardware-target migration, kernel debugging, performance optimization, operator fusion

• Preferred experience in both NVIDIA GPU and custom-accelerator ecosystems

• Background in compiler engineering, MLIR, or intermediate-representation lowering is advantageous

• Strong understanding of memory-hierarchy optimization is preferred

• Contributions to kernel or accelerator libraries such as cuBLAS, cuDNN, Triton community kernels, or JAX/XLA custom calls are advantageous

• Excellent written communication skills and the ability to provide precise technical feedback

• Must perform work without utilizing confidential or proprietary information from any employer, client, institution, or other third party

• H1-B and STEM OPT support is not available

🏝️ Benefits

• Opportunity for part-time independent contractor engagement

• Fully remote work within the United States

• Flexible scheduling based on project requirements

• Project duration may be extended, shortened, or concluded depending on project needs and performance

• H1-B and STEM OPT support is unavailable for this engagement