Senior Application Security Tester, AI Red Team Subject Matter Expert

This is a fully remote position, open to applicants in United States.

📋 Description

• The Senior Application Security Tester & AI Red Team Subject Matter Expert is a senior-level offensive security position for an expert who has excelled in modern web and API security and is now shaping how Evolve Security evaluates AI-enabled applications, large language models, and agentic systems.

• This role encompasses two key functions: serving as a hands-on senior application penetration tester for our most intricate client projects, and acting as the firm-wide subject matter expert who develops, expands, and represents Evolve Security’s AI red team practice.

• The senior tester conducts assessments with complete autonomy, manages the technical relationship with client security and engineering leadership, mentors mid-level engineers and OSOC analysts, and is recognized as the internal authority on offensive AI/ML testing methodologies, tools, and threat modeling.

⛳️ Requirements

• **Typical Experience:** 5–8+ years of offensive security experience with a strong focus on web application and API penetration testing, alongside proven hands-on experience in testing AI/ML systems — including LLM-backed applications, RAG pipelines, fine-tuned models, multi-agent systems, or production ML inference. A history of numerous completed assessments, published research, conference presentations, CVEs, or contributions to open-source projects is anticipated.

• **Domain Expertise:** Proficiency in web application and API security extending beyond the OWASP Top 10 — encompassing business logic abuse, intricate authentication and authorization flows (OAuth 2.0 / OIDC, SAML, JWT, mTLS), SSRF chains, deserialization, request smuggling, prototype pollution, and contemporary SPA / GraphQL attack surfaces. Equally knowledgeable in the OWASP Top 10 for LLM Applications and OWASP ML Top 10 — including prompt injection (direct, indirect, multi-modal), jailbreaks and safety bypasses, insecure output handling, training data poisoning and extraction, model denial of service, supply chain vulnerabilities in model and plugin ecosystems, excessive agency in agentic systems, sensitive data leakage from system prompts and embeddings, and vector store / RAG poisoning.

• **Technical Skills:** Mastery of the modern offensive toolchain — including Burp Suite Pro (with custom extensions), OWASP ZAP, Nuclei, Postman, Nmap, Metasploit, BloodHound — and the ability to create custom tools when existing solutions are inadequate. Proficient with AI red-teaming tools such as Garak, PyRIT, Promptfoo, Giskard, and adversarial ML libraries, and skilled in designing custom evaluation harnesses for client-specific LLM and agent stacks. Strong scripting and small-tool development capabilities in Python, with a working knowledge of JavaScript / TypeScript, Bash, and PowerShell. Familiarity with components of modern AI applications: vector databases (Pinecone, Weaviate, pgvector), embedding models, retrieval pipelines, agent frameworks (LangChain, LlamaIndex, CrewAI), and tool-use protocols including MCP.

• **Soft Skills:** Exceptional written and verbal communication skills — produces publication-quality reports without the need for editorial revisions, leads briefings for CISOs and engineering leaders, and effectively resolves contested findings with technical precision. Guides mid-level engineers and OSOC analysts through code reviews, paired testing, and methodology coaching. Comfortable representing Evolve Security externally — including webinars, podcasts, conference CFPs, and client thought-leadership content.

• **Certifications (Preferred, not required):** OSWE, OSCP, OSEP, GWAPT, GXPN, Burp Suite Certified Practitioner; AI/ML-adjacent credentials and contributions such as AI Red Team certifications, published prompt injection research, MITRE ATLAS contributions, or SANS SEC545/SEC595.

• Expertise that aligns with our approach includes:

• - Leading comprehensive web application and API penetration tests as the senior technical owner, scoping the engagement, executing the assessment, and delivering findings to client security and engineering leadership.

• - Applying structured testing techniques aligned with OWASP WSTG and OWASP API Security Top 10 to evaluate authentication, session management, access control (vertical and horizontal privilege escalation), input validation, error handling, and business logic vulnerabilities.

• - Designing and conducting AI red team engagements against LLM-backed applications, RAG systems, and agentic workflows — addressing prompt injection (direct, indirect, multi-modal), jailbreak resilience, system prompt and tool-use exfiltration, training data and embedding leakage, insecure output handling, and excessive agency in tool-using agents.

• - Mapping AI findings to the OWASP Top 10 for LLM Applications, OWASP ML Top 10, MITRE ATLAS, and the NIST AI Risk Management Framework, enabling client stakeholders to justify severity and remediation decisions internally.

• - Testing the complete AI application surface: model endpoints, prompt and response pipelines, retrieval augmentation, vector stores, fine-tuning pipelines, plugin / tool integrations (including MCP servers), guardrail and safety layers, and supporting cloud infrastructure.

• - Demonstrating proficiency in manual exploit development for both classical web vulnerabilities (XSS, SQLi, SSRF, IDOR, CSRF, deserialization) and LLM-specific attacks (jailbreak chains, indirect prompt injection via RAG content, agent hijacking via crafted tool outputs).

• - Validating authentication mechanisms — OAuth, OIDC, SAML, MFA implementations, and JWT — and their extension into AI-specific surfaces such as agent identity, per-user tool scoping, and prompt-level authorization.

• - Assessing session management, secrets handling, and data-flow controls in AI applications, including the pathways through which user data is incorporated into prompts, logs, vector stores, and model fine-tunes.

• - Executing client-side testing with browser development tools and proxy-based inspection, evaluating DOM-based vulnerabilities, insecure local storage, and AI-driven client behaviors (e.g., embedded copilots and in-page agents).

• - Testing REST and GraphQL APIs using a combination of dynamic, manual, and automated methods, applying the same rigor to model and agent APIs.

• - Performing code-assisted (grey-box) and full source review when available, identifying logic flaws, insecure configurations, and dangerous patterns specific to AI integrations (untrusted-content-into-prompt, unbounded tool use, missing output sanitization).

• - Building, maintaining, and contributing to Evolve Security’s AI red team methodology, payload libraries, evaluation harnesses, and reporting templates — while serving as the firm-wide reviewer for AI-related findings.

• - Mentoring mid-level penetration testing engineers and OSOC analysts through paired testing, technical reviews, knowledge-sharing sessions, and contributions to internal training and the academy.

• - Representing Evolve Security externally through conference presentations, blog posts, webinars, and client thought-leadership content focused on application security and AI red teaming.

• - Communicating findings clearly, emphasizing business impact, reproducibility, and strategic remediation guidance that engineering teams can effectively implement.

• Success in the first six months will look like:

• - A published, version-controlled AI red team methodology focused on LLM applications, RAG systems, and agentic workflows, adopted across Evolve Security engagements.

• - A reusable AI red team toolkit (custom Garak/PyRIT probes, payload libraries, evaluation harnesses) ready for any tester to utilize in client engagements.

• - Senior technical ownership of at least one strategic, AI-focused client account.

• - A mentorship cadence established with mid-level engineers and OSOC analysts, demonstrating an increase in their AI-related findings and reporting quality.

• - At least one piece of public thought leadership (talk, blog, or research) attributed to Evolve Security.

🏝️ Benefits

• Who is Evolve Security?

• Evolve Security is a cybersecurity services firm based in Chicago, IL. We are committed to enhancing our clients' security posture through continuous penetration testing, training services, and talent solutions.

• In addition to our professional cybersecurity service offerings, Evolve Security provides a cybersecurity bootcamp, “Evolve Academy”, which is currently ranked as the #1 cybersecurity bootcamp in the world. The Cybersecurity Bootcamp in Chicago offers immersive training, equipping students with the practical skills essential for the job. Students gain real-world experience through live security assessments conducted for non-profit organizations.

• We are passionate about directly enhancing our customers’ security posture, and we take pride in training others to meet the demand for qualified cybersecurity talent.

• Benefits Include:

• - Healthcare Benefits

• - 401(k) Match

• - Parental Leave

• - Flexible Paid Time Off

• - Annual vacation reimbursement