This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments of conversational AI models and agents through jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulations.

• Create human data by annotating failures, categorizing vulnerabilities, and identifying systemic risks.

• Utilize taxonomies, benchmarks, and playbooks to ensure consistent testing practices.

• Generate reproducible reports, datasets, and attack scenarios for clients.

• Analyze AI outputs related to sensitive topics, including bias, misinformation, and harmful behaviors.

• Identify vulnerabilities that automated testing may overlook.

• Broaden evaluation coverage and minimize unexpected outcomes in production.

• Engage in projects focused on training and enhancing advanced AI systems.

⛳️ Requirements

• Native proficiency in English and Assamese is essential.

• Strong discernment regarding language and content quality.

• Capability to evaluate whether AI responses are accurate, complete, and suitable.

• Proficiency in articulating reasoning clearly to both technical and non-technical audiences.

• Meticulous attention to detail, including subtle errors, inconsistencies, and gaps.

• Ability to consistently adhere to guidelines, taxonomies, benchmarks, playbooks, and quality standards.

• Flexibility to adapt across various projects, task types, and client needs.

• Must be an independent contractor.

• Currently, we are unable to accommodate H1-B or STEM OPT candidates.

• Preferred qualifications include expertise in adversarial ML, cybersecurity, socio-technical risk, conversational AI testing, psychology, acting, and writing.