Senior Principal AI Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Establish the technical vision, architecture, and multi-year roadmap for LLM and agentic systems within Detection and Response and Exposure and Remediation.

• Develop a comprehensive company-wide model strategy that encompasses model selection, fine-tuning, distillation, post-training, and investment in emerging capabilities.

• Oversee intricate, high-risk technical initiatives across various teams, focusing on the fine-tuning, distillation, and post-training of open-source LLMs utilizing security data.

• Create and implement company-wide standards for context engineering, inference optimization, and evaluation, with evaluations acting as the release gate for AI systems.

• Spearhead the optimization of LLM inference at scale, ensuring a balance between quality, latency, and cost.

• Conduct design reviews and manage complex technical trade-offs.

• Prototype and develop essential model, training, and inference code.

• Mentor and nurture the growth of Principal, Staff, and Senior engineers.

• Represent Rapid7’s AI engineering efforts to executives, customers, and the broader community.

• Influence product, platform, and business strategy while shaping strategic investments.

⛳️ Requirements

• 14+ years of experience in software or ML engineering.

• 2+ years of experience in developing production LLM or agentic AI systems.

• A PhD in Computer Science, Machine Learning, AI, or a related field is preferred, or equivalent expertise demonstrated through industry experience and technical contributions.

• Proven track record of building scalable, production-grade AI systems and shared platforms utilized across various products.

• Cutting-edge knowledge in LLMs, including transformer architecture, attention mechanisms, tokenization, and positional encodings.

• Advanced skills in PyTorch, covering custom model code, mixed precision, and distributed training, including FSDP and DeepSpeed.

• Hands-on experience in fine-tuning and post-training of open-weight LLMs, utilizing both full and parameter-efficient methods such as LoRA and QLoRA.

• Familiarity with distillation and preference or RL-based methods like DPO and GRPO.

• Extensive experience in context engineering, retrieval, memory management, and evaluation strategies for LLM and agent systems.

• In-depth knowledge of inference optimization techniques, including prefill and decode, KV-cache management, continuous batching, speculative decoding, and quantization.

• Experience in deploying models at scale using vLLM, SGLang, or TensorRT-LLM, preferably on AWS.

• Experience with GPU kernel development using CUDA or Triton is preferred.

• Strong foundational knowledge in predictive AI, classical ML, distributed systems, and Python.

• Recognized as an authority in AI through publications, patents, open-source contributions, or industry involvement.

• Capacity to make effective and transparent decisions among predictive AI, classical ML, and LLM methodologies based on factors such as accuracy, latency, and cost.

🏝️ Benefits

• Variable/incentive compensation.

• Equity.

• Benefits (where applicable/eligible).