CUDA Engineer

This is a fully remote position, open to applicants in Albania.

📋 Description

• Investigate and enhance the performance of LLM inference on contemporary GPUs

• Collaborate with inference and serving frameworks such as vLLM, SGLang, TensorRT-LLM, or equivalent systems

• Examine GPU execution, memory access patterns, kernel performance, and identify inference bottlenecks

• Develop, optimize, debug, or reverse-engineer CUDA kernels and fundamental GPU operations

• Investigate optimization strategies for matrix multiplication, attention mechanisms, and neural network layers

• Benchmark performance and discover avenues to enhance latency, throughput, and GPU utilization

• Tackle technically demanding challenges at the convergence of AI systems and high-performance computing

• Grasp GPU architecture, low-level kernels, LLM inference engines, and performance limitations

⛳️ Requirements

• Extensive hands-on experience with CUDA and GPU programming

• Profound understanding of GPU architecture, memory hierarchy, parallel execution, and performance enhancement

• Background in LLM inference, AI systems, GPU kernels, or high-performance computing

• Acquainted with vLLM, SGLang, TensorRT-LLM, Triton, or FlashAttention

• Experience troubleshooting intricate performance issues and examining low-level system behavior

• Strong analytical and problem-solving abilities

• Capability to convey complex technical concepts in English

• PhD or research experience in Computer Science, Machine Learning, Computer Architecture, or a related discipline (preferred)

• Publications, research endeavors, or significant contributions to open-source GPU/LLM infrastructure (preferred)

• Experience with PyTorch internals, distributed inference, quantization, KV cache optimization, or custom GPU kernels (preferred)

• Contributions to vLLM, SGLang, Triton, FlashAttention, or similar repositories (preferred)

🏝️ Benefits

• Security through client vetting to reduce risks and ensure reliable, timely payments

• Career support and assistance in finding new project opportunities

• Legal guidance regarding independent contractor or sole proprietorship status, taxes, and related processes

• English language courses

• Opportunities for professional growth

• Team-building activities

• Flexible working hours

• 29 days of paid time off (18 working days each year plus all national holidays)

• 10 paid recovery days

• Comprehensive financial and legal support for independent contractors

• Free English classes with native speakers or Ukrainian instructors

• Dedicated HR support