Research Scientist – Performance Optimization

This is a fully remote position, open to applicants in Europe.

📋 Description

• Analyze and enhance GPU, CPU, and accelerator code to ensure optimal utilization and minimal latency.

• Create high-performance code using PyTorch, Triton, and CUDA, including developing custom operations when necessary.

• Develop fused kernels and utilize tensor cores along with modern hardware capabilities across various platforms.

• Enhance model architectures and implementations for distributed multi-node production deployment.

• Create tools for performance monitoring, analysis, and automation.

• Investigate and apply state-of-the-art optimization techniques for transformer models.

• Profile existing training and inference processes within the first 30 days.

• Deliver and validate a kernel or architecture optimization during the 30 to 60-day period.

• Establish monitoring and automation to mitigate performance regressions in the 60 to 90 days timeframe.

⛳️ Requirements

• Advanced expertise in Triton/CUDA programming and GPU optimization.

• Proficient in PyTorch, particularly in kernel development and custom operations.

• Familiarity with profiling tools, such as NVIDIA Nsight, torch profiler, and custom tooling.

• Comprehensive knowledge of transformer architectures and attention mechanisms.

• Experience with compilers and exporters like torch.compile, TensorRT, ONNX, or XLA (preferred).

• Background in optimizing inference workloads for both latency and throughput (preferred).

• Understanding of Triton compiler and kernel fusion techniques (preferred).

• Knowledge of warp-level intrinsics and advanced CUDA optimization techniques (preferred).

🏝️ Benefits

• Equal opportunity employer.

• Participation in a voluntary diversity and inclusion survey; opting out will not impact the job application.