ML Engineer – AI Platform Lead

This is a fully remote position, open to applicants in Hong Kong.

📋 Description

• Implement and enhance a large language model for production inference, focusing on quantization, continuous batching, and low-latency serving.

• Develop the RAG pipeline, which includes document chunking, embedding generation, vector storage, cross-encoder reranking, and context assembly optimized for a 128K-token context window.

• Create the context layer, which features per-tenant system prompts, dynamically retrieved few-shot examples, and task routing to classify incoming requests to the appropriate prompt configuration.

• Establish defensive output parsing to produce structured JSON outputs from an unmodified base model with reliable fallbacks.

• Design and execute the feedback collection pipeline to capture user corrections and ratings, automatically generating training data candidates for future fine-tuning.

• Formulate the custom model training workflow that includes tenant-scoped LoRA training on client-specific data, model evaluation, A/B testing, and isolated deployment.

• Oversee and enhance inference quality by tracking parsing failure rates, citation accuracy, hallucination rates, and latency on a per-tenant basis.

• Collaborate with the domain expert daily to iterate on prompts during the pilot phase.

⛳️ Requirements

• A minimum of 5 years in ML engineering, with at least 2 years of experience working with large language models in production environments.

• Practical experience with LLM serving frameworks such as vLLM, TGI, or similar.

• Extensive experience in building RAG pipelines, including chunking strategies, embedding models, vector databases, and reranking.

• Proficient prompt engineering skills for production applications—capable of making a base model generate consistent, structured, and high-quality outputs.

• Proficient in Python, with experience in PyTorch, Transformers, and FastAPI.

• Familiarity with LoRA/QLoRA fine-tuning workflows.

🏝️ Benefits

• Opportunity to work on cutting-edge AI technologies with a dynamic team.

• Flexible work environment that encourages innovation and creativity.

• Competitive salary and performance-based bonuses.

• Professional development and training opportunities.

• Comprehensive health benefits and wellness programs.