This is a fully remote position, open to applicants in Colombia.

📋 Description

• Recognize realistic failures in search-related tasks where an AI model fails to deliver satisfactory responses.

• Develop and assess search queries.

• Record instances of failure.

• Compose guidelines and evaluation standards for accurate answers.

• Assess factors such as local relevance, source credibility, information timeliness, language alignment, answer thoroughness, and region-specific relevance.

• Conduct multiple tests on queries to identify failures.

⛳️ Requirements

• Research skills — ability to locate and authenticate information from various sources; adept at navigating diverse websites, databases, and documentation.

• Critical thinking — capable of detecting contradictions among sources, discerning outdated information, and understanding when something is not applicable to a specific locale or context.

• Attention to detail — proficient at identifying subtleties in language, local variations (such as prices, regulations, requirements), and edge cases.

• Problem-solving mindset — comfortable with iterating and testing when faced with challenges; remains unfazed by trial-and-error processes.

• Domain knowledge — preferably someone with experience in research, quality assurance, compliance, or similar fields that necessitate accuracy and source verification.

• Writing clarity — able to convey instructions and criteria clearly, ensuring others grasp the rationale behind their reasoning.

• Locale knowledge — having in-depth familiarity with their specific locale (including local regulations, business practices, and cultural nuances) is advantageous for identifying region-specific failures.

• Proficiency in Spanish (LATAM dialect).