Senior Data Scientist – Generative AI, Agents

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop and manage agents’ system prompts, which include establishing guardrails, permitted actions, flow control, tool-calling policies, output formats, and human escalation procedures.

• Organize and prepare knowledge bases for Retrieval-Augmented Generation (RAG), ensuring processes like cleaning, sectioning, validity metadata, and categorization by access groups.

• Assess retrieval quality and refine the RAG strategy accordingly.

• Create Optical Character Recognition (OCR) and information-extraction workflows for PDFs, images, and handwritten documents, incorporating quality metrics and solutions for illegible documents.

• Execute anonymization and personal data masking services, along with input and output guardrails to ensure data privacy.

• Develop golden datasets and a prompt evaluation framework, including regression testing for each version and measuring accuracy and cost per case.

• Analyze and approve Large Language Models (LLMs) from various providers, implementing AI FinOps practices.

• Collaborate with the technical team to create agents on the client’s platform, supporting integration testing, User Acceptance Testing (UAT), and assisted operations.

• Document technical decisions and provide supporting materials for the architecture committee, addressing costs, security, and data management.

• Participate in a project aimed at developing five AI agents to automate back-office processes for a client in the healthcare and pharmaceutical retail sector.

• Integrate agents with enterprise systems using APIs, Managed Cloud Platforms (MCPs), and Robotic Process Automation (RPA), facilitating interactions via Microsoft Teams.

• Work collaboratively in a squad alongside a solutions architect, developer, and business analyst.

⛳️ Requirements

• A Bachelor’s degree in Computer Science, Statistics, Mathematics, Engineering, Information Systems, or a related discipline.

• Over 4 years of experience in data science or machine learning, including involvement in Generative AI projects.

• Practical experience with LLMs: prompt engineering, function calling/tools, structured output (JSON Schema), and agent creation.

• Proficiency in RAG concepts: embeddings, vector databases, hybrid search, reranking, and retrieval assessment.

• Python skills for experimentation, evaluation, and data processing tasks.

• Experience in creating datasets and evaluation metrics for LLM solutions (accuracy, precision/recall, LLM-as-a-judge).

• Familiarity with OCR and document information extraction technologies (multimodal models and/or Document AI).

• Understanding of AI governance and Brazil’s LGPD, focusing on masking, anonymization, and data minimization practices.

• Intermediate to advanced proficiency in English.

• Preferred: Experience with Azure OpenAI and/or Microsoft Foundry; MongoDB Atlas Vector Search.

• Preferred: Knowledge of MCP protocol and agent patterns (ReAct, stateful workflows).

• Preferred: Understanding of LLM security measures: prompt injection, red teaming, and guardrails.

• Preferred: Experience with AI FinOps strategies, including estimating and optimizing token costs and evaluating different providers.

• Preferred: Proficiency in Node.js/TypeScript.

• Preferred: Experience with RPA or integrations with ServiceNow, Microsoft Teams/Graph, or SAP.

• Preferred: Business acumen in healthcare/regulatory affairs (ANVISA), supply chain management, or HR benefits.

• Preferred: Relevant cloud or AI certifications (e.g., Azure AI Engineer).

🏝️ Benefits

• Ongoing professional development opportunities.

• Potential for project extension or transition to permanent employment after project completion.

• Flexibility of remote work.