This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments of conversational AI models and agents by utilizing jailbreaks, prompt injections, misuse cases, bias exploitation, and multi-turn manipulation.

• Create human data through the annotation of failures, classification of vulnerabilities, and identification of systemic risks.

• Implement taxonomies, benchmarks, and playbooks to ensure consistency in testing.

• Generate reproducible reports, datasets, and attack scenarios.

• Evaluate AI outputs related to sensitive topics such as bias, misinformation, or harmful behaviors.

• Identify vulnerabilities that automated tests may overlook.

• Broaden evaluation coverage and minimize unexpected issues in production.

• Enhance customer AI systems through adversarial testing.

⛳️ Requirements

• Native fluency in English and Odia.

• Strong discernment regarding language and content.

• Capability to evaluate if AI responses are accurate, complete, and suitable, along with the ability to articulate the reasoning behind the assessment.

• Meticulous attention to detail in identifying subtle errors, inconsistencies, and omissions.

• Consistent adherence to guidelines and quality standards.

• Skill in clearly communicating reasoning to both technical and non-technical audiences.

• Flexibility to adapt across various projects, tasks, and client needs.

• Status as an independent contractor.

• H1-B and STEM OPT candidates are not eligible.

• Preferred specializations include adversarial machine learning, cybersecurity, socio-technical risk, and creative probing.