This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments on conversational AI models and agents utilizing jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulation.

• Create human-like data by annotating failures, categorizing vulnerabilities, and identifying systemic risks.

• Implement taxonomies, benchmarks, and playbooks to ensure consistency in testing.

• Generate reproducible reports, datasets, and attack scenarios.

• Evaluate AI outputs related to sensitive subjects such as bias, misinformation, or harmful behaviors.

• Identify vulnerabilities that automated tests may overlook.

• Broaden evaluation coverage while minimizing unexpected issues during production.

• Collaborate on initiatives aimed at training and improving AI systems.

⛳️ Requirements

• Native fluency in both English and Urdu is essential.

• Strong discernment regarding language and content.

• Capability to evaluate whether AI responses are accurate, comprehensive, and appropriate, along with the ability to articulate the reasoning behind assessments.

• Meticulous attention to detail regarding subtle errors, inconsistencies, and omissions.

• Consistent adherence to guidelines and quality standards.

• Proficiency in clearly explaining reasoning to both technical and non-technical audiences.

• Flexibility to adapt across various projects, task types, and client needs.

• Independent contractor status is required.

• Must not need H1-B or STEM OPT sponsorship.

• Preferred areas of expertise include adversarial machine learning, cybersecurity, socio-technical risk, or creative probing.