Senior Research Engineer

This is a fully remote position, open to applicants in New York.

📋 Description

• Enhance the experimental efficiency of the team by accelerating experiment launches, ensuring reliable measurements, and expediting next-step decisions.

• Oversee and advance the JAX training framework to facilitate scalable and efficient large-scale distributed TPU training.

• Explore model-data quality challenges, develop tools to highlight these issues, and convert insights into quantifiable accuracy improvements.

• Evaluate the accuracy of production models and create evaluation frameworks to pinpoint enhancements impacting customers.

• Convert research prototypes into production-ready solutions while refactoring and modernizing existing architectures and infrastructure.

• Improve production inference for speech language models through enhancements in serving architecture, quantization, and speculative decoding.

• Investigate and address performance limitations originating from low-level kernels such as XLA and Pallas, up to high-level system design.

• Collaborate with researchers, infrastructure teams, and production engineers to diagnose issues and implement sustainable solutions.

• Train models, conduct evaluations, and analyze data as required.

⛳️ Requirements

• Expert-level knowledge of JAX and TPUs, including proficiency with Flax, Optax, and the XLA compilation pipeline.

• Significant experience in optimizing inference systems for production, preferably with LLMs or speech models.

• Extensive knowledge of large-scale distributed training, modern deep learning systems, and best practices in ML infrastructure.

• Acquainted with continuous batching, KV-cache management, sharding techniques, and quantization.

• Strong proficiency in Python.

• Experience with C++ or Rust for kernel-level development is advantageous.

• Outstanding communication skills and a collaborative approach.

• Familiarity with Speech-to-Text domain knowledge, including ASR architectures, audio processing, and streaming inference, is a plus.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• A vibrant team culture with regular team-building activities.