This is a fully remote position, open to applicants in France.

📋 Description

• Recognize realistic failures related to search where an AI model falls short in its responses.

• Develop and assess search queries.

• Record instances where AI model responses were inadequate.

• Compose guidelines and evaluation criteria for accurate answers.

• Assess local relevance, source integrity, outdated information, language discrepancies, incomplete answers, and region-specific inapplicability.

• Conduct multiple query tests to identify failures.

⛳️ Requirements

• Research proficiency — capability to locate and verify information across various sources; comfortable navigating different websites, databases, and documentation.

• Critical thinking skills — able to detect inconsistencies between sources, pinpoint outdated information, and understand when something is not applicable to a particular locale or context.

• Detail-oriented — skilled at recognizing subtlety in language, local variations (prices, regulations, requirements), and edge cases.

• Problem-solving attitude — adept at iterating and testing when challenges arise; does not become frustrated with trial-and-error.

• Domain expertise — preferably someone with a background in research, quality assurance, compliance, or similar fields that demand accuracy and source validation.

• Clear writing skills — capable of expressing instructions and evaluation criteria in a manner that ensures understanding of the underlying logic.

• Local knowledge — extensive familiarity with their specific region (local regulations, business practices, cultural nuances) is advantageous for identifying region-specific failures.

• Proficiency in French.

• Availability for a commitment of 20 hours per week.