This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments on conversational AI models and agents by utilizing jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulation techniques.

• Create human data through the annotation of failures, classification of vulnerabilities, and identification of systemic risks.

• Implement taxonomies, benchmarks, and playbooks to ensure uniform testing practices.

• Generate reproducible reports, datasets, and attack scenarios for clients.

• Analyze AI outputs for bias, misinformation, and harmful behaviors.

• Detect vulnerabilities that automated tests may overlook.

• Broaden evaluation coverage and minimize unexpected issues in production.

• Support Mercor’s AI lab and enterprise clients in training and refining cutting-edge AI models.

⛳️ Requirements

• Required fluency or native proficiency in both English and Bengali.

• Strong discernment regarding language and content quality.

• Capability to evaluate AI responses for accuracy, completeness, and appropriateness, along with the ability to articulate the rationale behind these assessments.

• Meticulous attention to detail in identifying subtle errors, inconsistencies, and omissions.

• Consistent adherence to guidelines and quality standards.

• Ability to communicate reasoning effectively to both technical and non-technical audiences.

• Flexibility to adapt across various projects, task types, and client needs.

• Must qualify as an independent contractor.

• H1-B and STEM OPT candidates are not eligible.