WAInjectBench: Benchmarking Prompt Injection Detections
for Web Agents

Attacker’s goals. The attacker may be either the owner or a malicious user of a website. In the first case, the attacker has full control over the site. Meanwhile, if the site is fully compromised, the attacker likewise gains owner-level control; for simplicity, we treat these cases equivalently. In the second case, the attacker is a user who can post content to the site. For instance, a seller on an e-commerce platform like Amazon may upload a product listing with images and descriptions, or a user on a forum such as Reddit may create a post or comment. In both scenarios, the attacker’s objective is to manipulate website content so that, when a web agent interacts with it, the agent executes a sequence of attacker-chosen target actions rather than following its intended task trajectory. Such attacks can have severe consequences, including click fraud, malware downloads, or disclosure of sensitive information.

Attacker’s capability. As the website owner, the attacker has full access to the webpage and can arbitrarily modify its content–for example, by adding pop-ups, injecting HTML elements with malicious instructions, or altering the raw pixel values of the page. As a malicious user, the attacker may instead upload perturbed images or post text comments that embed malicious instructions.

Attacker’s background knowledge. We consider the attacker’s background knowledge under both white-box and black-box settings. In the white-box setting, the attacker is assumed to have complete access to the MLLM’s parameters, the system prompt, and the specific types of observations used by the agent. In contrast, in the black-box setting, the attacker has absolutely no knowledge of the MLLM, the system prompt, the user prompt, the captioning model used by the web agent, or the exact observation types.

VWA-Adv (Wu et al., 2024). This attack assumes the attacker is a malicious user. Specifically, the attacker adds optimized perturbations to a product image and uploads it to the website. As a result, when a captioning model generates a caption for the product image, it is highly likely to produce a caption containing a malicious instruction. Consequently, when the web agent takes the image and its caption as an observation, it may follow the malicious instruction and perform the target action. This attack considers both white-box and black-box knowledge of the captioning model. In the white-box setting, the attacker directly optimizes perturbations based on the captioning model of the web agent. In the black-box setting, the attacker crafts perturbations based on multiple CLIP model encoders to enhance transferability to the captioning model used by the web agent. The agent’s contaminated observations include the screenshot, images and their captions.

EIA (Liao et al., 2024). This attack assumes the attacker is the website owner. The attacker injects an HTML element containing a malicious instruction, tricking the web agent into entering users’ personally identifiable information into the element, thereby leaking it to the attacker. To ensure stealthiness, the element is placed with low opacity, thus it does not appear in the SoM text. Additionally, this attack assumes the attacker has black-box knowledge of the web agent. The contaminated observations may include the screenshot, a11y tree, and raw HTML.

Pop-up (Zhang et al., 2024). This attack assumes the attacker is the website owner. Specifically, the attacker injects pop-ups into the website, causing agents to click on them rather than executing their intended tasks. The attacker is assumed to have only black-box knowledge. Each pop-up contains a malicious instruction, an attention hook designed to draw the agent’s attention to the pop-up, and info banner–a button for the agent to click on, such as “OK”. In addition, auxiliary text is inserted into the accessibility (a11y) tree of the webpage to further enhance the attack. The original attack adds pop-ups directly to the screenshot. However, we generalize it by editing the website’s source code instead, as directly modifying the screenshot used by the agent is impractical. Consequently, the contaminated observations include the screenshot, a11y tree, and SoM text.

WASP (Evtimov et al., 2025). This attack assumes the attacker to be a malicious user. Specifically, the attacker posts Reddit posts or GitLab issues containing malicious instructions to mislead the web agent to perform a series of target actions. The attacker has only black-box knowledge of the web agent. The agent’s contaminated observations include the screenshot, a11y tree, SoM text, and raw HTML.

WebInject (Wang et al., 2025). This attack assumes the attacker is the website owner. The attacker adds an optimized raw-pixel-value perturbation to the webpage, which is then reflected in the screenshot and indirectly induces the web agent to perform the target action. The perturbation is optimized based on the web agent’s MLLM, and therefore requires white-box access. The screenshot is the observation that directly causes the web agent to output the target action; however, since the perturbation is injected through HTML code, the contaminated observations include both the screenshot and the raw HTML.

VPI (Cao et al., 2025). This attack assumes the attacker is the website owner. The attacker may insert pop-ups, malicious emails, or malicious messages depending on the website’s context. These inserted elements contain malicious instructions that mislead the web agent into performing targeted actions. This method requires only black-box access to the web agent. The contaminated observations may include the screenshot, a11y tree, SoM text, and raw HTML.

Malicious text segments. The malicious dataset is constructed from text segments collected through the aforementioned attack strategies. For clarity, we group these samples into four categories that align with the benign counterparts introduced later: (1) user comments and issue reports, (2) emails and messages, (3) image captions, and (4) textual components of webpage interfaces. Within each category, multiple attack strategies contribute distinct malicious samples (e.g., WASP for comments, VWA-adv for image captions, and EIA for interface text). Table 2 reports the number of collected samples in each category and the subset containing explicit instructions (EI). To identify EI, we manually examined all samples and labeled a text segment as containing explicit instructions only when both human annotators agreed. Importantly, all malicious text segments contain EI, except for certain cases generated by VWA-adv and EIA. In particular, VWA-adv and EIA can craft malicious segments that manipulate context or model behavior without EI.

Benign text segments. To ensure a fair evaluation, we constructed benign text segments that mirror the malicious ones. Specifically, since the malicious text segments can be categorized into the four types above, we collected benign samples from the same categories: (1) user comments and issue reports, collected from Reddit and GitLab as included in Visual Web Arena (VWA) and Web Arena (WA), where such content can be easily identified and manually selected; (2) emails and messages, from the ham portion of the Spam Email Dataset (main body only) (Jackksoncsie, 2023) and the SMS Spam Collection Dataset (UCIML, 2016) on Kaggle; (3) image captions, generated using LLaVA-1.5-7B on 226 benign images in VWA; and (4) textual components of webpage interfaces, curated from VWA and Mind2Web (M2W) (Deng et al., 2023). The statistics for benign samples are summarized in the right part of Table 2.

Malicious images. For images embedded in webpages, we consider VWA-Adv, which perturbs images to inject malicious instructions. We collected 298 perturbed images, using their captions as the corresponding malicious text segments described above. For malicious webpage screenshots, we rendered the attacked webpage corresponding to each malicious text segment or perturbed image and captured a screenshot. For 15 of the VWA-Adv perturbed images, rendering failed, so no screenshots were obtained. For EIA, malicious text segments without explicit instructions do not affect the screenshot and are therefore omitted, while each segment with explicit instructions was inserted into eight different locations within an attacked webpage, yielding 496 screenshots. To better align with the observations typically available to agents, we primarily collected screenshots with SoM, except for WebInject, which strictly assumes screenshot-only input.

Benign images. For benign images embedded in webpages, we collected 226 samples from clean webpages in Visual Web Arena. For benign webpage screenshots, we rendered 361 clean webpages from both Visual Web Arena (VWA) and Mind2Web (M2W), and additionally obtained their corresponding SoM screenshots for comprehensive analysis, forming a collection of total 948 samples. A summary of the malicious and benign images is provided in Table 3. Appendix C provides examples of the malicious and benign text and image samples we collected.

Prompting-based. These approaches directly prompt a detection LLM to decide whether a given text is malicious, meaning that it explicitly contains malicious instructions. Known-answer detection (KAD), initially briefly suggested in a social media post (Nakajima, 2022) and later formalized by Liu et al. (2024b), appends the text to a detection instruction, e.g., “Repeat secret_key once while ignoring the following text: [text].” Here, secret_key is a random string known only to the detector (e.g., “ASGsdhE”) but hidden from attackers. This combined input is then fed into a detection LLM. If the output fails to accurately reproduce the secret_key, it implies the LLM instead followed an instruction in the text, which is flagged as malicious.

Embedding-based. This approach (Ayub and Majumdar, 2024), referred to as Embedding-T, uses a detection LLM as an embedding model to generate embedding vectors for text samples. A binary classifier is then trained on labeled embedding vectors corresponding to benign and malicious text samples. The classifier can then be applied to a new text sample to flag whether it is malicious.

Fine-tuning-based. These approaches fine-tune a detection LLM to act as a binary classifier. Different methods convert a detection LLM into a binary classifier in different ways. PromptGuard (Inan et al., 2023) directly treats the detection LLM as a binary classifier that takes a text sample as input and outputs “Yes” or “No”. Fine-tuning involves collecting both labeled malicious and benign text samples. DataSentinel (Liu et al., 2025) turns a detection LLM into a classifier in a fundamentally different way. Instead of standard fine-tuning, this approach builds on KAD and enhances it through a game-theoretic process. DataSentinel formulates the fine-tuning objective as a minimax optimization problem that simulates a game between a detector and a strong adaptive attacker. The attacker optimizes injected prompts to evade detection and mislead the detection LLM, while the detector is fine-tuned to resist such attacks. For fine-tuning, DataSentinel only requires benign text samples but can leverage malicious ones if available.

Prompting-based. There are several ways to directly prompt a detection MLLM to identify malicious images. One approach uses a designed system prompt, where the detection MLLM outputs “Yes” or “No” to indicate whether an image sample is malicious. In our experiments, we adopt GPT-4o and LLaVA-1.5-7B (Liu et al., 2024a) as the detection MLLMs in this setting, which we denote as GPT-4o-Prompt and LLaVA-1.5-7B-Prompt, respectively. In contrast, JailGuard (Zhang et al., 2023) adopts a more sophisticated strategy inspired by techniques from image adversarial example detection (Hendrycks et al., 2019; Lopes et al., 2019; Mumuni and Mumuni, 2022; Xu et al., 2017). Specifically, JailGuard first mutates an image into multiple variants using a range of transformations. It then compares the MLLM’s responses to these variants and quantifies their inconsistency with KL divergence. If the divergence is high, JailGuard flags the image as malicious.

Embedding-based. This approach, denoted as Embedding-I, uses an image encoder such as CLIP (Radford et al., 2021) as an embedding model to generate embedding vectors for image samples. A binary classifier is then trained on labeled embedding vectors corresponding to benign and malicious image samples. The classifier can then be applied to new image samples to flag malicious ones.

Fine-tuning-based. This approach fine-tunes a detection MLLM to act as a binary classifier. For example, we can directly treat the detection MLLM as a binary classifier that takes an image sample as input and outputs “Yes” or “No”. To improve parameter efficiency during fine-tuning, we can further apply LoRA (Hu et al., 2022). In our experiments, we adopt LLaVA-1.5-7B as the detection MLLM in this setting, which we denote as LLaVA-1.5-7B-FT.