Independent Red Teaming and AI Safety Evaluation

This is a fully remote position, open to applicants in Kenya.

📋 Description

• Perform a concentrated and independent red-team assessment of WAZZII’s girl-focused generative AI capabilities.

• Identify safety, safeguarding, clinical, and technical vulnerabilities.

• Evaluate if the system identifies and appropriately escalates relevant risks.

• Determine if current evaluations and controls reflect practical usage.

• Create reproducible, machine-readable, and rerunnable test cases.

• Prioritize findings based on severity, likelihood, and potential impact.

• Offer actionable, prioritized recommendations for remediation and future evaluations.

• Review evaluation frameworks, red-line policies, and escalation procedures.

• Co-develop scenario taxonomy, severity rubric, and responsible-disclosure protocols.

• Execute both adversarial and realistic non-adversarial testing in a staging environment.

• Assess clinically incorrect or harmful health advice, safeguarding escalations, jailbreaks, prompt injections, policy circumventions, and age-inappropriate content.

• Test in English, Kiswahili, and Sheng, including code-switching patterns.

• Stress-test human escalation and moderation pathways.

• Generate an inception note and test plan by week 3.

• Deliver a severity-ranked findings report with reproducible test cases by week 8.

• Provide a leadership briefing and prioritized recommendations by week 10.

• Engage in weekly progress meetings.

• Maintain a shared delivery tracker and risk register.

• Utilize secure communication and evidence-sharing channels.

• Immediately escalate critical issues.

• Safeguard access credentials and evaluation materials.

• Independently manage personnel and delivery strategy.

⛳️ Requirements

• Independent organization or specialized consultancy.

• Independence is a crucial requirement; bidders must not develop, operate, or commercially benefit from the systems under evaluation.

• No existing contract to develop or operate the evaluated system.

• No financial interest in the evaluation reaching a specific outcome.

• Disclosure of current or past affiliations with Girl Effect’s AI, platform, or technology suppliers, including relevant relationships over the last 24 months.

• Disclosure of tools or models under evaluation in which the bidder has a financial stake.

• Proven experience in independently red-teaming deployed AI systems.

• Familiarity with conversational AI, retrieval-augmented generation, and multi-turn systems.

• Experience in health, safeguarding-sensitive, or other high-risk environments.

• Background in adolescent or youth-facing digital services.

• Strong understanding of AI safety, responsible AI, and model evaluation.

• Capability to develop automated, machine-readable, and rerunnable testing frameworks.

• Knowledge of human escalation and moderation workflows.

• Ability to test in English, Kiswahili, and Sheng either directly or through a credible quality-controlled method.

• Experience working in Kenya, Africa, or similar Global South contexts.

• Strong practices in security, confidentiality, and responsible disclosure.

• Exceptional analytical, technical-reporting, and leadership-presentation skills.

• Proposed team must consist of named individuals with expertise in AI evaluation, technical aspects, safeguarding, and language.

• Separate technical and financial proposals, limited to 10 pages.

• Three client references required.

• One redacted example of a relevant findings report to be provided.

• Successful bidder will undergo due diligence, reference checks, safeguarding, information-security, and compliance evaluations.

• Successful bidder must adhere to Girl Effect’s code of conduct, safeguarding, and anti-fraud standards.

• No significant subcontracting without prior written consent.

• Proposals must be valid for 90 days from the submission deadline.

🏝️ Benefits

• Girl Effect will grant approved access to the staging environment.

• Programmatic access to the conversational endpoint when available.

• Access to relevant documentation and evaluation materials.

• Designated technical, safeguarding, and program focal points.

• Consolidated feedback on deliverables.

• Payments will be linked to satisfactory completion and written acceptance of agreed deliverables.

• Reasonable expenses approved in writing in advance may be reimbursed.

• Warranty period for rectifying defects in accepted deliverables at no extra charge, as proposed.

• Opportunity for remote work; no in-country presence required.

• Girl Effect will hold ownership of foreground intellectual property created specifically during the engagement.