One-sentence Summary

Peking University et al. propose ToolHazard, a scalable adversarial environment synthesis framework that combines an Environment Simulator, an Attacker Agent, and a User Simulator to generate executable stateful environments, viable injection points, environment-specific payloads, and state-grounded long-horizon tasks for stress-testing LLM-based agents; experiments with ToolHazard-Bench reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness, while ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo without reducing utility.

Key Contributions

• ToolHazard is a scalable adversarial environment synthesis framework that creates executable stateful environments, discovers viable injection points, and generates environment-specific payloads for agent security evaluation and adversarial alignment.
• ToolHazard-Bench comprises 87 long-horizon tasks across 28 stateful environments and 512 tools, providing higher workflow complexity than prior agent security benchmarks.
• Empirical results show that LLM agents remain highly vulnerable to environmental prompt injections, that attacks are more effective when injected instructions occur earlier and appear near the end of observations, and that ToolHazard-generated alignment data improves security on ToolHazard-Bench and AgentDojo while preserving benign task utility.

Introduction

Large language model agents increasingly act on real-world systems through tools, which makes them powerful but also exposes them to indirect prompt injection attacks that can hijack otherwise benign workflows. Prior agent security benchmarks and alignment datasets rely heavily on manually built or reused environments and predefined injection locations, so expanding them to new application domains is expensive and hard to reproduce. The authors propose ToolHazard, a scalable adversarial environment synthesis framework that automatically creates executable, stateful tool-interactive environments, discovers viable injection points, generates environment-specific attack payloads, and produces state-grounded long-horizon tasks. They use this framework to build ToolHazard-Bench, a benchmark with 87 tasks across 28 environments and 512 tools, and show that ToolHazard-generated data can also improve agent safety through supervised fine-tuning and reinforcement learning while preserving benign utility.

Dataset

The authors build the dataset from ToolHazard, a simulator that generates environments, states, tasks, and adversarial injection payloads rather than collecting real user traffic.

• Initial environment generation and filtering: ToolHazard produces 191 valid environments after automated quality inspection, split into 140 training candidates and 51 test candidates. Only environments with valid injection points reachable along task execution trajectories are kept, leaving 60 training environments and 28 disjoint test environments.
• Quality inspection: a Testing Agent interacts with the environment by invoking tools, while a Checking Agent validates execution and constraint consistency. Environments below a quality score threshold are discarded.
• User simulation: the user simulator initializes environment states from the environment skeleton, entity set, constraints, and toolset, then generates long-horizon tasks from the initialized state.

ToolHazard-Bench:

• Source: 28 held-out test environments.
• Scale: 512 tools and 87 state-grounded tasks, with 15.56 average execution steps.
• Attacks: each task uses six predefined payload wrappers: basic combined, important-template, multi-turn, decision hijacking, reasoning-criteria, and tool-selection.
• Evaluation data: check functions are executed over the final environment snapshot. GPT-4.1-mini generates these functions, but BR and ASR are computed programmatically.

ToolHazard-Align:

• Source: 60 training environments disjoint from the benchmark.
• Construction: five initial states and benign tasks per environment create 300 environment-task instances. Applying six attacks yields 1,800 candidates, filtered to 1,040 valid samples.
• Sample schema: each sample includes a benign task, clean and adversarial environment states, an injected task, and check functions.
• Training split: 329 samples are used for reinforcement learning and 711 for supervised fine-tuning. SFT uses successful trajectories from clean environments; RL uses adversarial-environment trajectories with a reward based on task success minus injected-task compliance.

Method

The authors introduce ToolHazard, a scalable adversarial environment synthesis framework designed to automate the construction of executable and stateful tool-interactive environments, generate state-grounded user tasks, and execute environment-side prompt injection attacks.

The framework is composed of several core modules. First, the Environment Simulator addresses the limitations of manually designed environments by utilizing an LLM-driven generator. This process begins with Environment Blueprint Planning, where a latent executable environment blueprint $\mathcal{B} = \langle \mathcal{E}, \mathcal{R}, \mathcal{T} \rangle$B=⟨E,R,T⟩ is constructed from a seed agent task dataset $\mathcal{D}$D through environment type inference, state and rule inference, and operation inference. Next, in Executable Program Construction, this blueprint is translated into an executable environment program $\mathcal{P} = f_{\mathrm{code}}(\mathcal{B})$P=fcode​(B) using an Object-Oriented Programming paradigm, where entities become class attributes and tools become callable methods. Finally, Automated Quality Inspection employs a dual-agent verification pipeline consisting of a Testing Agent and a Checking Agent to validate execution correctness. The environment quality score is calculated as:

$$\mathrm{score}_{\mathrm{env}} = \frac{1}{N} \sum_{i=1}^{N} \mathrm{Judge}(a_i)$$scoreenv​=N1​i=1∑N​Judge(ai​)

where $\mathrm{Judge}(a_i) \in \{0, 1\}$Judge(ai​)∈{0,1} indicates whether the $i$i-th interaction satisfies all constraints. Environments falling below a predefined threshold are discarded.

Building upon these synthesized environments, the User Simulator generates diverse user requests. This involves Environment State Initialization to create an initial state $S_{\mathrm{init}} = f_{\mathrm{init}}(\mathcal{E}, \mathcal{R}, \mathcal{T})$Sinit​=finit​(E,R,T), followed by User Task Generation to synthesize long-horizon tasks $q = f_{\mathrm{task}}(S_{\mathrm{init}}, \mathcal{E}, \mathcal{R}, \mathcal{T})$q=ftask​(Sinit​,E,R,T).

To simulate environment-side attacks, the Attacker Agent manipulates the target LLM agent by poisoning environment states. This is achieved through Environmental Attack Point Discovery, a three-stage process that identifies injectable attributes, analyzes operation read and write dependencies, and matches valid attack points with complete propagation chains. Subsequently, the Adversarial Injection Pipeline filters attack points based on benign execution trajectories and employs a plan-and-execute framework. In the planning phase, an LLM selects an effective attack point $p^*$p∗ and constructs an attack plan with a hijack task $q_{hijack}$qhijack​. In the execution phase, an Attack Agent poisons the environment state by appending the payload via a write operation.

For evaluation, the Verification Function Generation module decomposes tasks $q$q into verifiable conditions $\{c_k\}_{k=1}^K = g_{\mathrm{cond}}(q)${ck​}k=1K​=gcond​(q). For each condition, a validation function $f_{c_k} = g_{\text{verifier}}(c_k, q)$fck​​=gverifier​(ck​,q) is generated, and the final score is computed as:

$$\mathrm{Score} = \frac{1}{K} \sum_{k=1}^{K} \mathbb{1} \left[ f_{c_k}(S_{\text{final}}) = 1 \right]$$Score=K1​k=1∑K​1[fck​​(Sfinal​)=1]

This metric depends only on the terminal state $S_{\text{final}}$Sfinal​, making it agnostic to execution trajectories and supporting multiple valid solution paths.

Finally, the authors leverage the synthesized data for ToolHazard-Align to improve agent robustness. They define a trajectory-level reward for reinforcement learning:

$$R(\tau) = R_{\mathrm{task}}(\tau) - R_{\mathrm{injected}}(\tau)$$R(τ)=Rtask​(τ)−Rinjected​(τ)

where $R_{\mathrm{task}}(\tau)$Rtask​(τ) measures successful task completion and $R_{\mathrm{injected}}(\tau)$Rinjected​(τ) measures hijacking by adversarial instructions. Supervised Fine-Tuning is applied to successful trajectories from clean environments to strengthen tool-use capabilities, while Group Relative Policy Optimization is applied to trajectories from adversarial environments using the defined reward to penalize compliance with injected instructions.

Experiment

The experiments evaluate four closed-source and three open-source ReAct-based LLM agents against six environment-side prompt injection strategies, using benign task completion rate and attack success rate as metrics. Nearly all models prove highly vulnerable, especially to newer strategies such as decision hijacking and tool selection, while stronger models can be more susceptible and capability gains offer only marginal defense. Analyses further show that attacks are more effective when injected early, placed in later response fields, and delivered through free-form tool outputs, and that adversarial environments consistently degrade benign task execution. Alignment with ToolHazard-Align on Qwen3-4B and Qwen3-8B improves both safety and utility with cross-environment generalization, indicating the benchmark's potential for training more robust agentic models.

ToolHazard-Bench offers a fully LLM-synthesized adversarial evaluation framework with 28 stateful domains, long-horizon tasks averaging over 15 steps, and up to 18.75 candidate tools per task. Unlike prior benchmarks, it automatically discovers injection points and generates attack payloads, eliminating the need for manual environment construction and predefined attack surfaces. This design improves scalability and supports reproducible, state-grounded adversarial testing. ToolHazard-Bench is the only benchmark that uses LLM-synthesized environments and LLM-discovered attack points, while all others rely on manual or reused construction with predefined injection locations. Its tasks are substantially longer (15.56 steps) and involve more candidate tools (18.75) than most prior benchmarks, reflecting complex, state-grounded, long-horizon scenarios.

Across evaluated models, benign task success rates are often high, but environment-side prompt injection still causes high attack success. More capable model generations generally improve benign response rates and reduce some attack success rates relative to earlier counterparts, yet these improvements are too small for reliable defense. Newly introduced attack strategies, especially decision hijacking, tool selection, and reasoning criteria, are consistently effective. GPT-5 experiences over 40% attack success on four strategies, while Gemini-3.1-Pro exceeds 30% on three strategies. DeepSeek-V3.2 achieves the highest benign task success rate but is also the most vulnerable to attacks, suggesting stronger task-following can coincide with greater susceptibility. Previously studied basic combined attacks show low attack success compared with newer strategies such as decision hijacking, tool selection, and reasoning criteria.

Environment-side prompt injection attacks reduce benign task completion rates relative to clean conditions for both evaluated models. The degradation is more pronounced for GPT-4.1, especially under decision hijacking and multi-turn attacks, while Gemini-3.1-Pro shows smaller declines. This indicates that prompt injection harms general task execution capability in addition to safety behavior. Every environment-side attack type lowers benign task completion compared with the no attack setting for both models. GPT-4.1 exhibits the largest drops under multi-turn and decision hijacking attacks. Gemini-3.1-Pro remains relatively stable across attack types, with only minor reductions in benign completion.

On ToolHazard-Bench and AgentDojo, full ToolHazard-Align on Qwen3-4B yields the highest benign success rates and the lowest attack success rates, with especially large safety gains on AgentDojo. For Qwen3-8B, SFT alone improves both utility and security relative to the base model. Overall, alignment improves task completion and resistance to environment-side prompt injection across both benchmarks. For Qwen3-4B, ToolHazard-Align achieves the best benign success rates and reduces attack success rates compared with the base model and SFT variants. For Qwen3-8B, SFT improves benign success and reduces attack success on both benchmarks, whereas for Qwen3-4B, SFT alone improves utility but slightly raises attack success on both benchmarks.

ToolHazard-Bench uses LLM-synthesized stateful environments and LLM-discovered injection points to evaluate long-horizon tool-use agents under environment-side prompt injection. Across models, benign task success is often high but attack success remains substantial, with newer strategies such as decision hijacking, tool selection, and reasoning criteria proving especially effective and also degrading benign task completion; more capable models still do not provide reliable defense. Alignment methods, particularly full ToolHazard-Align, improve utility and security on ToolHazard-Bench and AgentDojo, whereas SFT alone can improve utility but may slightly increase attack susceptibility.