Doctoral Research Fellow – LLM, Model Evaluation, Python, Prompt Engineering

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Create and execute evaluations and tests of synthetic personas to replicate various user profiles, objectives, and behaviors.

• Build experimental LLM-as-a-Judge pipelines, which involve prompt engineering, rubric development, and automated evaluation protocols to facilitate scalable analysis of AI-generated outputs.

• Integrate concepts from Reinforcement Learning and Preference Optimization.

• Investigate and implement preference optimization strategies and reinforcement learning to enhance language models continuously, assessing their effects on quality, robustness, and alignment.

⛳️ Requirements

• Education: PhD.

• Fields of study: Computer Science, Computer Engineering, Information Systems, Data Science, Statistics, Applied Mathematics, or related domains in Computing, Artificial Intelligence, or Machine Learning.

• Programming expertise: Python.

• Understanding of Machine Learning and Deep Learning principles.

• Familiarity with Large Language Models (LLMs) and Generative AI.

• Experience with AI model development frameworks (PyTorch and/or Hugging Face Transformers).

• Capability to read and comprehend scientific literature in English.

• Basic understanding of experimental design, results analysis, and model evaluation.

• Experience with LLM evaluation / LLM-as-a-Judge (preferred).

• Experience with RLHF, DPO, or preference optimization (preferred).

• Involvement in scientific research/publications in AI or Machine Learning (preferred).

• Familiarity with code agents (Claude Code, GitHub Copilot, Codex, etc.) (preferred).

🏝️ Benefits

• Stipend: R$ 11,000.00.

• Remote work.