Senior Principal Software Scientist

This is a fully remote position, open to applicants in Massachusetts.

📋 Description

• Design and develop large-scale transformer and hybrid foundation models.

• Take ownership of model architecture decisions across text, multimodal, and novel paradigms.

• Identify and resolve training instabilities on a large scale.

• Assess scaling tradeoffs among data, computational resources, and architecture.

• Establish the technical roadmap for next-generation models.

• Utilize strong principles in deep learning and representation learning.

• Create and adapt transformer architectures, including Attention variants, RoPE, ALiBi, Grouped Query Attention (GQA), and Mixture-of-Experts (MoE).

• Construct models from foundational principles rather than merely modifying existing codebases.

• Take responsibility for optimizer and scheduler selections, such as AdamW, Lion, and Adafactor.

• Grasp and troubleshoot optimizer instability, gradient pathologies, and divergence on a large scale.

• Validate scaling laws and assess the tradeoffs between compute and data.

⛳️ Requirements

• Strongly Required: Profound theoretical and practical knowledge of modern deep learning.

• Hands-on experience in training large models from the ground up.

• Ability to reason about optimization beyond merely tuning hyperparameters.

• Comfort in navigating ambiguous, research-oriented environments.

• Critical Technical Skills: Understanding of transformer internals and attention mechanisms.

• Knowledge of optimization algorithms and training dynamics.

• Familiarity with scaling laws and compute/data tradeoffs.

• Experience with distributed training strategies and mixed precision.

• Innovation in architecture for large, real-world models.

🏝️ Benefits

• Opportunity for an annual bonus.

• Comprehensive insurance coverage (medical, dental, vision, life, and disability).

• Paid time off.

• Paid holidays.

• Company contributions to the RRSP (Registered Retirement Savings Plan).

• Equity awards available for specific roles and levels.

• Flexibility for remote and/or hybrid work arrangements based on the position.