Senior ML Specialist

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Take ownership of model strategy across the platform by assessing and selecting models for each task.

• Utilize benchmarks and LLM-as-judge frameworks with statistically robust analysis to inform model decisions.

• Develop and uphold evaluation methodologies, which include offline evaluation sets, judge calibration, regression benchmarks, and quality metrics.

• Create classification and fine-tuned models for routing, categorization, and detection tasks.

• Construct datasets, as well as train, evaluate, and deploy models.

• Design, develop, and deliver agentic AI products, which encompass agent identities, reusable skills, tool integrations, and structured outputs.

• Execute prompt and context engineering, which includes assembling system prompts, injecting live-data context, managing threads/memory, and producing Pydantic structured outputs.

• Ensure output quality using deterministic enforcers, validation quality gates, and LLM-as-judge evaluators.

• Identify hallucinations, inconsistencies, and model-version drift, implementing solutions through prompts, retrieval, guardrails, or model selection.

• Build and enhance agent tools that query Databricks, MongoDB, and other external systems.

• Connect agents with the larger ecosystem via the Model Context Protocol (MCP).

• Manage deployed services using OpenTelemetry, monitoring quality, latency, and cost in Grafana.

• Work collaboratively with data engineers, software engineers, and product teams on comprehensive AI solutions.

• Elevate the team's machine learning fundamentals through reviews and knowledge sharing.

• Rigorously assess research literature and apply validated methodologies.

• Document and maintain the codebase, ensuring high standards of code quality and best practices.

⛳️ Requirements

• Over 5 years of practical experience in machine learning, including training, evaluating, and deploying models.

• Strong foundation in statistics and probability, encompassing hypothesis testing, confidence intervals, sampling, sample-size reasoning, bias/variance, and calibration.

• Profound understanding of LLM and generative AI principles, including transformers, attention mechanisms, tokenization, embeddings, training pipelines, decoding strategies, scaling behaviors, and potential failure modes.

• Direct experience with classical machine learning and natural language processing, including classification, clustering, feature engineering, text classification, embeddings, and semantic similarity.

• Proficient in fine-tuning models using LoRA/PEFT, supervised fine-tuning, or full fine-tuning; capable of constructing training datasets.

• Experience in designing AI evaluation methodologies, including offline evaluation sets, LLM-as-judge, inter-rater agreement, regression benchmarks, and significance testing.

• Production-level Python proficiency, including PyTorch, scikit-learn, Hugging Face, modern asynchronous Python, FastAPI, Pydantic, testing, and continuous integration.

• Practical experience with at least one major LLM provider API: OpenAI, Anthropic, or Google, including structured outputs and tool/function calling.

• A Master's or PhD in Machine Learning, Statistics, Computer Science, Mathematics, or a related quantitative discipline, or equivalent demonstrated expertise through publications, competition results, released models, or research code.

• Ability to work autonomously at a senior level, taking ownership of problems end to end.

• Preferred: Publications or contributions to NLP/ML; a record of competitions on Kaggle or equivalent platforms.

• Preferred: Experience with retrieval systems, agent evaluation, model cost/latency optimization, Databricks, Azure, Docker, observability stacks, MCP, and 0-to-1 product delivery.

🏝️ Benefits

• An equal opportunity employer dedicated to diversity, inclusion, and a sense of belonging.

• PJ contract arrangement.