AI Research Engineer

This is a fully remote position, open to applicants in Czechia, +2 more countries.

📋 Description

• Design, construct, and sustain automated execution pipelines for ongoing LLM benchmarking across a variety of languages, tasks, and models.

• Develop and implement LLM-assisted pre-annotation and quality assurance pipelines for the annotation of production data.

• Execute and enhance LLM-as-a-judge pipelines, tailored evaluation metrics, and scoring algorithms.

• Establish experimental infrastructure, datasets, and execution pipelines for multilingual LLM research studies and thought leadership reports.

• Convert applied NLP research into scalable prototypes and integrate research tools into production operations alongside the data engineering team.

• Collaborate on research insights, benchmark reports, and thought leadership publications.

• Prototype emerging AI capabilities including agentic AI evaluation, synthetic data quality certification, and domain-specific LLM assessments.

• Maintain documentation, code repositories, and technical guides for evaluation frameworks and tools.

⛳️ Requirements

• Master’s degree (MSc or equivalent) in Computer Science, Artificial Intelligence, Computational Linguistics, or a related technical discipline.

• 3–5+ years of practical experience in NLP or ML engineering.

• Advanced expertise in Python and deep learning/NLP frameworks, such as PyTorch or TensorFlow and Hugging Face transformers.

• Demonstrated experience with LLM orchestration frameworks like LangChain or LlamaIndex.

• Familiarity with automated evaluation methods such as LLM-as-a-judge and benchmarking harnesses.

• Proficient in data manipulation, structured evaluation data processing, and analysis using pandas, NumPy, or similar tools.

• Hands-on experience deploying and managing ML workflows on platforms like Azure, AWS, or Vertex AI.

• Experience in building scalable data or experiment orchestration pipelines.

• Knowledge of multilingual NLP, machine translation, or computational linguistics.

• Understanding of localization workflows, annotation platforms, or human-in-the-loop data pipelines.

• Proven track record of contributing to open-source NLP projects, evaluation frameworks, or published technical benchmarks.

🏝️ Benefits

• Equal opportunity employment and a workplace free from discrimination and harassment.

• Opportunities for personal growth and career advancement.

• An inclusive environment that values diversity.