Senior Deep Learning Software Engineer, Inference, Model Optimization

This is a fully remote position, open to applicants in California.

📋 Description

• Train, develop, and deploy cutting-edge generative AI models, including LLMs and diffusion models, utilizing NVIDIA's AI software stack.

• Utilize and enhance the Torch 2.0 ecosystem, featuring TorchDynamo, torch.export, and torch.compile, to analyze and derive standardized model graph representations from various Torch models.

• Create high-performance inference optimization techniques, such as automated model sharding, tensor parallelism, sequence parallelism, and efficient attention kernels with KV-caching.

• Collaborate with cross-functional teams within NVIDIA to incorporate efficient kernel implementations into the automated deployment solution.

• Analyze and evaluate GPU kernel-level performance to uncover optimization opportunities in both hardware and software.

• Drive continuous innovation in inference performance for NVIDIA's TRT, TRT-LLM, and TRT Model Optimizer software solutions.

• Design and architect a modular, scalable software platform that supports extensive model coverage and optimization techniques.

⛳️ Requirements

• Master's degree, PhD, or equivalent experience in Computer Science, AI, Applied Mathematics, or a related field.

• Over 8 years of relevant professional or research experience in Deep Learning.

• Exceptional software design capabilities, including debugging, performance analysis, and test design.

• Strong expertise in Python, PyTorch, and associated machine learning tools (e.g., HuggingFace).

• Solid understanding of algorithms and programming fundamentals.

• Effective written and verbal communication skills, with the ability to work both independently and collaboratively in a dynamic environment.

• Contributions to PyTorch, JAX, or other Machine Learning Frameworks.

• Familiarity with GPU architecture and the compilation stack, along with the ability to understand and troubleshoot end-to-end performance.

• Experience with NVIDIA's deep learning SDKs, such as TensorRT.

• Previous experience in writing high-performance GPU kernels for machine learning tasks using frameworks such as CUDA, CUTLASS, or Triton.

🏝️ Benefits

• Highly competitive salaries.

• Comprehensive benefits package.

• Equity.