AI Data Quality Analyst – Human-in-the-Loop

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Assess AI-extracted data from insurance submissions (SOVs, loss runs, supporting documents) for accuracy, completeness, and consistency.

• Compare the extracted fields with source documents, identify discrepancies, and amend data directly within the relevant systems or templates.

• Serve as a quality checkpoint for the AI pipeline, ensuring the output aligns with established business and underwriting expectations before it proceeds downstream.

• Record issues, defects, and edge cases with detailed reproduction steps, examples, and impact assessments, utilizing tools like Jira or similar platforms.

• Detect patterns and root causes of extraction errors (e.g., recurring issues with specific formats, document types, or fields).

• Convert observed patterns into well-organized requirements, user stories, and bug reports that engineering and data teams can utilize.

• Work collaboratively with architects, data engineers, and other analysts to enhance extraction rules, templates, and workflows.

• Utilize LLMs and AI assistants as resources (e.g., for summarization, cross-checking, hypothesis generation), while exercising sound judgment regarding trust and verification.

• Aid in the ongoing improvement of documentation, checklists, and guidelines for reviewing submissions and extractions.

• Gradually contribute to establishing metrics and dashboards for data quality and model performance (e.g., accuracy by field, error rates by document type).

⛳️ Requirements

• Over 3 years of experience in a data-centric role, such as data analyst, business analyst, QA analyst, operations analyst, or a related position.

• Exceptional attention to detail and a proven track record of performing systematic, repetitive data reviews without compromising quality.

• Strong analytical skills: ability to trace issues from symptoms (incorrect numbers, missing fields) back to probable root causes (document patterns, parsing logic, business rules).

• Proven capability to compose clear, structured tickets/requirements for engineering teams (e.g., bug reports, user stories, acceptance criteria).

• Advanced proficiency in Excel (pivot tables, lookups, filters, data cleansing techniques).

• Comfortable working with complex business documents and datasets (financial, insurance, or similarly structured data).

• Strong written English skills for documenting findings, drafting tickets, and communicating with distributed teams.

• Able to handle repetitive tasks: reviewing a high volume of documents/records daily while maintaining consistency and care.

• Experience with issue-tracking or project management tools (e.g., Jira, Azure DevOps, Trello, or similar).

• Capable of working independently, managing your own workload, and escalating appropriately when patterns or blockers arise.

🏝️ Benefits

• Health insurance

• Retirement plans

• Paid time off

• Flexible work arrangements

• Professional development