Software Engineer 4/5 – Model Serving Systems, AI Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and enhance the compute infrastructure to meet Netflix's expanding AI requirements.

• Create scalable and resilient systems tailored for AI/ML applications.

• Offer infrastructure that supports real-time model inference and serving.

• Develop foundational abstractions to ensure uniformity between online and offline systems.

• Establish model-serving infrastructure for large language models (LLMs) and other significant foundation models.

• Facilitate the integration of machine learning in new business sectors.

• Propel AI/ML innovation throughout Netflix.

• Collaborate across functions with engineers, product managers, machine learning engineers, and data/research scientists.

• Ensure business-critical models maintain high availability and performance.

• Contribute to production hosting, deployment management, performance optimization, and capacity planning.

⛳️ Requirements

• Proven experience in creating high-traffic distributed services and infrastructure for online ML model inference.

• Knowledge of supporting large-scale ML models with an emphasis on high availability and performance.

• Comprehension of scalable model-serving solutions for generative models and LLMs.

• Expertise in minimizing latency and costs while addressing bottlenecks to enhance research-to-production workflows.

• Proficiency in object-oriented programming, with a preference for Java.

• Commitment to engineering excellence in production hosting, including performance optimization, deployment management, and capacity planning.

• Familiarity with deploying ML models using Triton Inference Server, TensorRT, and Docker.

• Experience with public cloud platforms such as AWS, Azure, or GCP.

• Understanding of observability and logging best practices.

• Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related discipline.

🏝️ Benefits

• Health Plans

• Mental Health support

• 401(k) Retirement Plan with employer match

• Stock Option Program

• Disability Programs

• Health Savings and Flexible Spending Accounts

• Family-forming benefits

• Life and Serious Injury Benefits

• Paid leave of absence programs

• Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off

• Full-time salaried employees are immediately entitled to flexible time off