One-sentence Summary

Johns Hopkins University, NVIDIA, UC Berkeley et al. propose EvoSafeHarness, a framework that automatically synthesizes model- and domain-specific safety harnesses by jointly evolving a natural-language policy and executable code with adversarial screening, achieving a superior safety–utility frontier across four agent benchmark families, reducing attack success rate from $45.6\%$45.6% to $10.0\%$10.0% on DecodingTrust-Agent with only a $3.3$3.3-point utility loss, reaching $82.8\%$82.8% utility at $0.0\%$0.0% ASR on AgentDojo, and maintaining mean ASR below $20\%$20% under adaptive PAIR attacks.

Key Contributions

• EvoSafeHarness automatically synthesizes a safety harness for a frozen LLM agent by jointly searching a natural-language policy and executable code logic, tailored to the specific model and target domain rather than reusing a fixed expert design.
• The optimization process incorporates behavioral feedback from the target model, a domain specification that captures required safety relations, and a fresh-context adversarial review that discards rules overfitted to benchmark artifacts.
• Across four agent benchmark families, the synthesized harnesses establish a stronger safety-utility frontier than fixed expert defenses. On DecodingTrust-Agent, average attack success rate drops from 45.6% to 10.0% at a 3.3-point utility cost; on AgentDojo, utility reaches 82.8% at 0.0% ASR, twice that of CaMeL at the same zero-ASR point; and mean ASR stays below 20% under adaptive PAIR attacks.

Introduction

As language-model agents move into production, handling sensitive data and critical systems, safety failures can cause concrete harm such as unauthorized transactions or data leaks. The threat surface is broad: adversaries can embed malicious instructions in external content (indirect prompt injection) or issue harmful commands directly through the user channel. While model-level safety training helps, it does not guarantee system-level enforcement, and existing harness-level defenses are typically fixed, expert-designed mechanisms that apply the same policy across different models and application domains. This one-size-fits-all approach struggles because the ideal enforcement strength depends on the model's own safety behavior, and domain-specific risks require tailored safety relations and control flow. Naively automating harness design with a scalar reward can lead to degenerate solutions like refusing all actions or overfitting to benchmark artifacts. The authors introduce EvoSafeHarness, a safety-focused meta-harness that searches for a model- and domain-specific defense by iteratively proposing and refining a natural-language policy along with executable code. The search loop incorporates a fresh-context critic, a cascade test environment, and decomposed feedback on benign, direct-attack, and indirect-attack outcomes to avoid common pitfalls, producing harnesses that significantly improve the safety-utility trade-off across diverse benchmarks.

Method

The authors introduce EvoSafeHarness, an automated search loop designed to discover deployment-specific security harnesses for tool-using agents. Rather than relying on a fixed set of defense mechanisms, the system treats harness design as an open-ended optimization problem over both natural-language policies and executable code logic.

As illustrated in the framework diagram, the system orchestrates a continuous cycle of proposal, critique, evaluation, and analysis.

The framework operates by iteratively proposing, critiquing, and evaluating candidate harnesses $H = (P, C)$H=(P,C), where $P$P represents the natural-language policy applied to the model context and $C$C denotes the executable code logic that mediates tool interactions. The primary objective is to maximize a scalar score that balances benign utility $U$U and attack success rate $\text{ASR}$ASR:

This formulation ensures that security improvements are only rewarded when the core functionality of the agent remains intact.

The search process begins with a warm-start phase, where the system distills design experiences from mature existing defenses into initial candidates and reusable guidance. This provides robust starting invariants, such as treating tool outputs as untrusted data, without constraining the search to a predefined template.

At the core of the loop is the Designer module, which selects a parent harness from an explicit archive and proposes free-form edits. The archive serves as the optimizer state, retaining every completed candidate, design hypotheses, and failure traces. The Designer can revise the policy $P$P, introduce trajectory state, modify control flow, or rewrite the code logic $C$C entirely, subject only to the application adapter constraints.

Before a candidate enters the evaluation pipeline, it undergoes a rigorous review by the Criticizer. Operating in a fresh and independent context, the Criticizer challenges the proposal with benchmark-independent evasions to prevent the search from overfitting to specific benchmark artifacts. It enforces a generalization requirement, ensuring that rules rely on attack invariants like provenance, scope, and intent mismatch rather than literal token matching. Flagged trivial evasions are repaired before consuming expensive evaluation budgets.

Surviving candidates are then passed to the Cascade Test Environment, which employs a sequential and nested evaluation strategy to conserve resources. The cascade consists of four stages: a static structural check, a smoke prefix test, a confidence gate that prunes statistically inferior candidates, and a full search extension. This staged approach separately measures benign utility and both direct and indirect attack success rates, ensuring that weak candidates are discarded early.

Finally, the Analyzer module processes the failed traces generated during evaluation. It distills these failures into summarized experiences, which are fed back into the archive to inform the subsequent iterations of the Designer. This closed-loop mechanism allows the system to progressively refine its understanding of the victim model vulnerabilities and the specific security boundaries of the target domain.

Experiment

The experiments evaluate EvoSafeHarness on DecodingTrust-Agent, Agent-SafetyBench, AgentCanary, and AgentDojo-to-AgentDyn across multiple victim models and domains, comparing against fixed baselines. Per-deployment search consistently improves the safety-utility trade-off, handles harms beyond prompt injection, transfers to unseen environments without re-search, and withstands adaptive attacks. Analysis shows that the harness discovers model- and domain-specific safety relations, moving from open-world content classification to closed-world action authorization, and that the evaluation cascade reduces inner evaluation cost without changing the selected defense.

Across five victim models and three application domains, the adaptive EvoSafeHarness defense reduces average attack success rate from 45.6% to 10.0% while preserving 79.8% benign utility, a drop of only 3.3 points. Fixed baselines like CaMeL and DRIFT are domain-dependent, helping on os-filesystem but failing on telecom, and the strongest fixed baseline, Progent, nearly matches the safety gain but at a much larger utility cost (56.4%). The defense is effective against both direct and indirect prompt injections, with the sole exception being a model-domain cell where undefended risk was already extremely low. Undefended attack success rates span an order of magnitude across models, from under 5% for Sonnet 4.6 to over 70% for DeepSeek-V4-Flash, making model-agnostic defenses brittle. EvoSafeHarness achieves the best safety-utility trade-off in 14 of 15 cells, lowering direct ASR from 50.9% to 12.6% and indirect ASR from 40.4% to 7.4% while retaining most benign functionality.

On the held-out Agent-SafetyBench split, the EvoSafeHarness defense achieves the lowest unsafe-behavior and attack-success rates for DeepSeek V3.2 while preserving utility, unlike Progent which reduces safety metrics but severely degrades utility. The Kimi-K2.6 model exhibits lower inherent unsafe behavior and higher utility without any defense, illustrating that the safety-utility trade-off of a harness is model-dependent. EvoSafeHarness reduces DeepSeek V3.2's unsafe-behavior rate from 28.3 to 14.6 and attack success rate from 23.8 to 9.4, while utility under attack rises to 54.3. Kimi-K2.6 without defense already has an unsafe-behavior rate of 21.7 and utility of 66.7, outperforming most defended configurations of DeepSeek V3.2, highlighting model-dependent harness effectiveness.

EvoSafeHarness sharply reduces attack success rates across all domains and attack sources, with telecom indirect attacks nearly eliminated. Residual risk concentrates in OS-filesystem and finance, which together account for 98 of the 105 remaining successful attacks. Certain risk types like client-targeted scams show no reduction, revealing a gap against harms based on fabricated claims rather than observable policy violations. OS-filesystem direct attacks remain the largest residual risk, leaving 43 successes (24.6% ASR) after defense. Finance is the only domain where indirect attacks have a higher residual ASR than direct attacks (13.7% vs. 9.7%).

The evaluation spans five victim models and three application domains, comparing the adaptive EvoSafeHarness defense against fixed baselines under both direct and indirect prompt injection attacks. EvoSafeHarness substantially lowers attack success rates while retaining most benign utility, achieving the best safety-utility trade-off in nearly all tested configurations, whereas baselines like CaMeL and DRIFT are domain-dependent and Progent sacrifices far more utility. On a held-out benchmark, the defense is effective but model-dependent, as some models already exhibit low unsafe behavior without any harness. Residual risk concentrates in OS-filesystem and finance domains, and certain harm types such as client-targeted scams show no reduction, highlighting a gap against fabricated-claim attacks.