This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments on conversational AI models and agents through methods such as jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulations.

• Create human-generated data by annotating failures, categorizing vulnerabilities, and identifying systemic risks.

• Implement taxonomies, benchmarks, and playbooks to ensure consistent testing.

• Generate reproducible reports, datasets, and attack scenarios.

• Evaluate AI outputs related to sensitive subjects, including bias, misinformation, and harmful behaviors.

• Investigate vulnerabilities that automated testing may overlook.

• Enhance evaluation coverage and minimize unexpected challenges in production.

• Fortify customer AI systems through adversarial testing.

⛳️ Requirements

• Required native fluency in both English and Punjabi.

• Strong discernment regarding language and content.

• Capability to evaluate whether AI responses are accurate, complete, and suitable, along with the ability to articulate the reasoning behind these assessments.

• Keen attention to subtle errors, inconsistencies, and gaps.

• Consistent adherence to guidelines and quality standards.

• Ability to communicate reasoning clearly to both technical and non-technical audiences.

• Flexibility to adapt across various projects, task types, and customer needs.

• Engagement as an independent contractor is required.

• Must not require H1-B or STEM OPT sponsorship.

• Preferred experience in adversarial machine learning, cybersecurity, socio-technical risk, or creative probing.