Inference Optimization Engineer

This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Develop the optimization platform that enhances the inference performance of LLMs deployed on Modular Cloud utilizing the latest GPU and ASIC architectures.

• Determine the technical trajectory of Modular Cloud to achieve LLM performance on the Pareto frontier for agentic applications.

• Collaborate with the GTM team to provide tailored LLM inference aligned with customer use cases.

• Work together with engineering teams on optimizations that span GPU kernels and cloud infrastructure.

• Convert insights from customer interactions into technical guidance for engineering teams.

• Write blog posts highlighting innovative methods for optimizing LLM inference.

• Participate in 2-4 team events and meetups annually.

⛳️ Requirements

• Over 5 years of experience in distributed systems or performance engineering.

• Proven history of creating durable, reusable software tools and libraries that are utilized across various teams and functions.

• Strong judgment in assessing technical trade-offs and prioritizing tasks.

• Excellent communication and technical leadership abilities.

• A creative and inquisitive approach to tackling complex problems.

• A collaborative and team-oriented attitude.

• Alignment with Modular’s organizational culture.

• Candidates must reside in the U.S. or Canada.

• Preferred but not mandatory: experience with GPU kernel programming, inference engine internals, or distributed inference architectures.

• Preferred but not mandatory: familiarity with Kubernetes and cloud-native ecosystems.

• Preferred but not mandatory: knowledge of modern LLM architectures and the latest techniques in inference optimization.

• Must have authorization to work in the U.S. for employment within the U.S.

🏝️ Benefits

• Comprehensive healthcare coverage.

• Retirement and savings programs.

• Opportunities for employee stock purchases.

• Paid time off.

• Resources for wellbeing.

• Family support initiatives.

• Opportunities for learning and development.

• RSU grants.

• Regular team onsite events and local meetups.

• Travel 2-4 times a year.