This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team assessments of conversational AI models and agents through jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulations.

• Create human-generated data by annotating failures, classifying vulnerabilities, and identifying systemic risks.

• Utilize taxonomies, benchmarks, and playbooks to maintain consistency in testing.

• Generate reproducible reports, datasets, and attack cases for clients.

• Evaluate AI outputs related to sensitive issues, including bias, misinformation, or harmful behaviors.

• Assist in broadening evaluation coverage and enhancing the resilience of customer AI systems.

⛳️ Requirements

• Proficiency in both English and Punjabi is essential.

• Strong analytical skills regarding language and content; ability to evaluate the accuracy, completeness, and appropriateness of AI responses, along with the capacity to articulate the reasoning.

• Keen attention to detail to identify subtle errors, inconsistencies, and omissions.

• Consistent adherence to guidelines and quality standards.

• Capability to convey reasoning effectively to both technical and non-technical audiences.

• Flexibility to adapt to various projects, task types, and client needs.

• Must have independent contractor status.

• H1-B and STEM OPT candidates are not eligible.

• Preferred qualifications include expertise in adversarial machine learning, cybersecurity, socio-technical risk, and creative probing.