Senior Application Security Tester, AI Red Team Subject Matter Expert

📋 Description

• The Senior Application Security Tester & AI Red Team Subject Matter Expert is a high-level offensive security position for a tester who has achieved expertise in contemporary web and API security, and is now establishing the framework for how Evolve Security evaluates AI-enabled applications, large language models, and agentic systems. This dual-role involves being a hands-on senior application penetration tester for our most intricate client projects, as well as the firm's subject matter expert who develops, expands, and represents Evolve Security’s AI red team initiative. The senior tester conducts assessments with complete independence, maintains the technical relationship with client security and engineering leadership, mentors mid-level engineers and OSOC analysts, and is acknowledged as the internal authority on offensive AI/ML testing methodologies, tools, and threat modeling.

⛳️ Requirements

• **Typical Experience:** 5–8+ years of offensive security experience with a strong focus on web application and API penetration testing, alongside demonstrable hands-on experience in testing AI/ML systems — including LLM-backed applications, RAG pipelines, fine-tuned models, multi-agent systems, or production ML inference. A proven history of completing numerous assessments, publishing research, delivering conference talks, contributing to CVEs, or engaging in open-source projects is anticipated.

• **Domain Expertise:** Comprehensive knowledge of web application and API security extending beyond the OWASP Top 10 — including business logic abuse, complex authentication and authorization flows (OAuth 2.0 / OIDC, SAML, JWT, mTLS), SSRF chains, deserialization, request smuggling, prototype pollution, and modern SPA / GraphQL attack surfaces. Proficient in the OWASP Top 10 for LLM Applications and OWASP ML Top 10 — covering prompt injection (direct, indirect, multi-modal), jailbreaks and safety bypasses, insecure output handling, training data poisoning and extraction, model denial of service, supply chain vulnerabilities in model and plugin ecosystems, excessive agency in agentic systems, sensitive data leakage from system prompts and embeddings, and vector store / RAG poisoning.

• **Technical Skills:** Mastery of the contemporary offensive toolchain — including Burp Suite Pro (with custom extensions), OWASP ZAP, Nuclei, Postman, Nmap, Metasploit, BloodHound — and the capability to create custom tools when standard options are insufficient. Comfortable using AI red-teaming tools such as Garak, PyRIT, Promptfoo, Giskard, and adversarial ML libraries, with confidence in designing tailored evaluation harnesses for client-specific LLM and agent stacks. Strong scripting and small-tool development skills in Python, with familiarity in JavaScript / TypeScript, Bash, and PowerShell. Knowledgeable about components of modern AI applications: vector databases (Pinecone, Weaviate, pgvector), embedding models, retrieval pipelines, agent frameworks (LangChain, LlamaIndex, CrewAI), and tool-use protocols including MCP.

• **Soft Skills:** Exceptional written and verbal communication — produces high-quality reports that require no editorial revisions, leads briefings for CISOs and engineering leaders, and effectively de-escalates contested findings with technical rigor. Guides mid-level engineers and OSOC analysts through code reviews, paired testing, and methodological training. Comfortable representing Evolve Security in external settings — including webinars, podcasts, conference CFPs, and client thought-leadership content.

• **Certifications (Preferred, not required):** OSWE, OSCP, OSEP, GWAPT, GXPN, Burp Suite Certified Practitioner; AI/ML-related credentials and contributions such as AI Red Team certifications, published prompt injection research, MITRE ATLAS contributions, or SANS SEC545/SEC595.

• Expertise that aligns with our methodology:

• - Lead comprehensive web application and API penetration tests as the senior technical owner, scoping engagements, executing assessments, and presenting findings to client security and engineering leadership.

• - Implement structured testing methods in line with OWASP WSTG and OWASP API Security Top 10 to evaluate authentication, session management, access control (vertical and horizontal privilege escalation), input validation, error handling, and business logic vulnerabilities.

• - Design and execute AI red team engagements targeting LLM-backed applications, RAG systems, and agentic workflows — addressing prompt injection (direct, indirect, multi-modal), jailbreak resilience, system prompt and tool-use exfiltration, training data and embedding leakage, insecure output handling, and excessive agency in tool-using agents.

• - Correlate AI findings with the OWASP Top 10 for LLM Applications, OWASP ML Top 10, MITRE ATLAS, and the NIST AI Risk Management Framework to assist client stakeholders in justifying severity and remediation decisions internally.

• - Assess the complete AI application surface: model endpoints, prompt and response pipelines, retrieval augmentation, vector stores, fine-tuning pipelines, plugin/tool integrations (including MCP servers), guardrail and safety layers, and supporting cloud infrastructure.

• - Demonstrate expertise in manual exploit development for classical web vulnerabilities (XSS, SQLi, SSRF, IDOR, CSRF, deserialization) as well as LLM-specific attacks (jailbreak chains, indirect prompt injection via RAG content, agent hijacking through crafted tool outputs).

• - Validate authentication mechanisms — OAuth, OIDC, SAML, MFA implementations, and JWT — and their application to AI-specific surfaces such as agent identity, per-user tool scoping, and prompt-level authorization.

• - Evaluate session management, secrets handling, and data-flow controls in AI applications, including how user data is utilized in prompts, logs, vector stores, and model fine-tunes.

• - Conduct client-side testing employing browser developer tools and proxy-based inspection, analyzing DOM-based vulnerabilities, insecure local storage, and AI-driven client behaviors (e.g., embedded copilots and in-page agents).

• - Test REST and GraphQL APIs through a blend of dynamic, manual, and automated techniques; apply the same thoroughness to model and agent APIs.

• - Perform code-assisted (grey-box) and complete source reviews when available, identifying logical flaws, insecure configurations, and risky patterns associated with AI integrations (untrusted-content-into-prompt, unbounded tool use, missing output sanitization).

• - Develop, maintain, and contribute to Evolve Security’s AI red team methodology, payload libraries, evaluation harnesses, and reporting templates — serving as the firm-wide reviewer for AI-related findings.

• - Guide mid-level penetration testing engineers and OSOC analysts through paired testing, technical evaluations, knowledge-sharing sessions, and contributions to internal training and the academy.

• - Represent Evolve Security publicly through conference presentations, blog posts, webinars, and client thought-leadership content on application security and AI red teaming.

• - Clearly communicate findings, emphasizing business impact, reproducibility, and strategic remediation advice that engineering teams can effectively implement.

• Success in the initial 6 months includes:

• - A published, version-controlled AI red team methodology that encompasses LLM applications, RAG systems, and agentic workflows, adopted across Evolve Security projects.

• - A reusable AI red team toolkit (custom Garak/PyRIT probes, payload libraries, evaluation harnesses) available for any tester to utilize during client engagements.

• - Senior technical oversight of at least one key AI-focused client account.

• - A structured mentorship approach with mid-level engineers and OSOC analysts, showing measurable improvement in their AI-related findings and reporting standards.

• - At least one piece of publicly attributed thought leadership (talk, blog, or research) associated with Evolve Security.

🏝️ Benefits

• Who is Evolve Security?

• Evolve Security is a cybersecurity services firm based in Chicago, IL. Our mission is to enhance our clients’ security posture by offering ongoing penetration testing, training services, and talent solutions.

• Alongside our professional cybersecurity service offerings, Evolve Security operates a cybersecurity bootcamp, “Evolve Academy”, which is currently recognized as the #1 cybersecurity bootcamp globally. The Cybersecurity Bootcamp in Chicago provides immersive training, equipping students with the concrete and practical skills necessary for the job. Participants gain real-world experience through live security assessment projects conducted for non-profit organizations.

• We are committed to directly enhancing our customers’ security posture, and we take pride in training others to fulfill the growing demand for qualified cybersecurity professionals.

• Benefits Include:

• - Healthcare Benefits

• - 401(k) Match

• - Parental Leave

• - Flexible Paid Time Off

• - Annual vacation reimbursement