Senior AI Scientist

This is a fully remote position, open to applicants in California.

📋 Description

• Design and develop large-scale transformer and hybrid foundation models.

• Take ownership of model architecture decisions across text, multimodal, and emerging paradigms.

• Identify and rectify training instabilities at scale.

• Assess scaling trade-offs involving data and compute resources.

• Utilize robust fundamentals in deep learning and representation learning.

• Construct models from the ground up, rather than merely adapting existing codebases.

• Make decisions regarding optimizers and schedulers, including AdamW, Lion, and Adafactor.

• Create and test loss functions such as next-token prediction and contrastive objectives.

• Plan and implement large-scale training utilizing FSDP, ZeRO-3, Tensor parallelism, and Pipeline parallelism.

• Implement mixed precision (bf16, fp8) and gradient checkpointing techniques.

⛳️ Requirements

• Design and develop large-scale transformer and hybrid foundation models.

• Take ownership of model architecture decisions across text, multimodal, and emerging paradigms.

• Identify and rectify training instabilities at scale.

• Assess scaling trade-offs involving data and compute resources.

• Utilize robust fundamentals in deep learning and representation learning.

• Design and modify transformer architectures, including Attention variants, RoPE, ALiBi, Grouped Query Attention (GQA), and Mixture of Experts (MoE).

• Construct models from the ground up.

• Make decisions regarding optimizers and schedulers, including AdamW, Lion, and Adafactor.

• Understand and troubleshoot optimizer instability and gradient pathologies.

• Apply and validate scaling laws.

• Create and test loss functions such as next-token prediction and contrastive objectives.

• Plan and implement large-scale training utilizing FSDP, ZeRO-3, Tensor parallelism, and Pipeline parallelism.

• Implement mixed precision (bf16, fp8) and gradient checkpointing techniques.

• Collaborate closely with ML systems teams.

🏝️ Benefits

• Annual bonus opportunity.

• Comprehensive insurance coverage (medical, dental, vision, life, and disability).

• Paid time off.

• Paid holidays.

• Company contribution to the RRSP (Registered Retirement Savings Plan).

• Equity awards available for specific positions and levels.

• Options for remote and/or hybrid work depending on the position.