AI Research Engineer – Pre-training, LLM, Multi-Modal

This is a fully remote position, open to applicants in United Arab Emirates (UAE).

📋 Description

• Execute foundational pre-training for LLMs and Multi-Modal models (integrating text, vision, audio, or other modalities) on extensive, distributed servers featuring multi-nodes and thousands of NVIDIA GPUs.

• Conceptualize, prototype, and scale innovative architectures, tokenizers, and cross-modal alignment layers to enhance model intelligence and multi-modal comprehension.

• Acquire, filter, and curate large-scale textual and multi-modal datasets, establishing robust data pipelines for efficient pre-training.

• Independently and collaboratively conduct experiments, analyze outcomes, and refine training methods for optimal performance and token efficiency.

• Explore, debug, and resolve bottlenecks in model efficiency, computational performance, and multi-modal alignment stability during prolonged training sessions.

• Contribute to the evolution of distributed training systems to ensure seamless scalability and hardware efficiency on designated platforms.

⛳️ Requirements

• A degree in Computer Science or a related discipline.

• Ideally, a PhD in NLP, Machine Learning, or a related area, supported by a strong track record in AI R&D (with notable publications in A* conferences).

• Practical experience contributing to large-scale LLM or Multi-Modal pre-training efforts on extensive, distributed servers equipped with thousands of NVIDIA GPUs, ensuring scalability and significant advancements in model performance.

• Familiarity and hands-on experience with large-scale, distributed training frameworks, libraries, and tools.

• In-depth understanding of state-of-the-art transformer and non-transformer modifications aimed at enhancing intelligence, efficiency, and scalability.

• Strong expertise in PyTorch and Hugging Face libraries, with practical experience in model development, continual pretraining, and deployment.

🏝️ Benefits

• Flexible working hours.

• Opportunities for professional development.