This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct red-team evaluations of conversational AI models and agents through methods such as jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulation.

• Create human data by annotating failures, classifying vulnerabilities, and identifying systemic risks.

• Utilize taxonomies, benchmarks, and playbooks to ensure consistent testing.

• Generate reproducible reports, datasets, and attack scenarios that clients can implement.

• Evaluate AI outputs related to sensitive issues including bias, misinformation, or harmful behaviors.

• Assist in expanding evaluation coverage and enhancing the safety, robustness, and reliability of AI systems.

⛳️ Requirements

• Proficiency in English and Telugu is essential.

• Native-level fluency in both English and Telugu is required.

• Strong judgment regarding language and content; ability to assess the accuracy, completeness, and appropriateness of AI responses, and articulate the rationale behind evaluations.

• Capability to identify subtle errors, inconsistencies, and gaps.

• Consistency in adhering to guidelines and quality standards.

• Ability to communicate reasoning clearly to both technical and non-technical audiences.

• Flexibility to adapt across various projects, task types, and client needs.

• Engagement as an independent contractor.

• Candidates must not hold H1-B or STEM OPT status.

• Preferred specialties include adversarial ML, cybersecurity, socio-technical risk, or creative probing.