Research Engineer, Large Language Models

Posted 2 days ago

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design and conduct extensive pre-training and fine-tuning experiments, ranging from small-scale ablations to comprehensive training runs.

• Analyze scaling laws to determine model size, data composition, and hyperparameters while adhering to a fixed compute budget.

• Enhance training efficiency using parallelism techniques (data, tensor, pipeline, FSDP), mixed precision, and custom CUDA or Triton kernels.

• Identify and resolve training instabilities such as loss spikes, divergence, and hardware malfunctions.

• Develop and optimize data pipelines, which include filtering, deduplication, quality assessment, and dataset mixing.

• Apply concepts from recent research publications and evaluate their effectiveness at scale.

• Create assessments that accurately measure real model capabilities and meticulously document results.

• Collaborate closely with alignment, infrastructure, and product teams, sharing insights through well-structured write-ups.


⛳️ Requirements

• 3+ years of practical experience in training deep learning models, or a PhD in a relevant field (computer science, mathematics, or physics).

• Proficient in Python with extensive knowledge of PyTorch or JAX.

• Hands-on experience with Transformer architectures, including attention mechanisms, tokenization, embeddings, and positional encodings.

• Familiarity with distributed training on multi-node GPU clusters (FSDP, DeepSpeed, Megatron-LM, or similar technologies).

• Strong foundational knowledge in linear algebra, probability, statistics, and optimization techniques.

• Capability to interpret a research paper, implement it from the ground up, and evaluate its effectiveness.

• Meticulous experimental practices, including maintaining clean baselines, conducting ablations, achieving reproducible results, and tracking experiments.

• Excellent written and verbal communication skills, along with the ability to thrive in a fast-paced and ambiguous environment.


🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional growth and development.

• Flexible work arrangements and a supportive team culture.

• Access to cutting-edge technologies and resources.

• Comprehensive health and wellness benefits.

People also viewed

Sedgwick2 days ago

Construction Defect Adjuster

US flagAlabama OnlyFull-timeUncategorized
ApplyView job
Cardinal Health2 days ago

Nurse, Individualized Care

US flagUnited States OnlyFull-timeUncategorized$68.5k – $107.6k/year
ApplyView job
BlueCross BlueShield of Tennessee2 days ago

Health Navigator

US flagTennessee OnlyFull-timeUncategorized
ApplyView job
Centene Corporation2 days ago

Care Navigator

US flagIllinois, +1 more stateFull-timeUncategorized$22 – $38/hour
ApplyView job
Centene Corporation2 days ago

Clinical Review Nurse – Concurrent Review

US flagFlorida OnlyFull-timeUncategorized$27 – $48/hour
ApplyView job
Worldwide Clinical Trials2 days ago

Clinician, Clinical Assessment Technologies

GB flagUnited Kingdom, +6 more countriesFull-timeUncategorized
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers