One-sentence Summary

Researchers from KAIST and DeepAuto.ai propose WikiTrace, an evidence-linked memory system that stores each lesson as a concise insight linked to a success/failure summary and the raw trajectory; it enables LLM agents to check, fill in, or correct lessons before acting and consistently outperforms existing memory approaches on five agentic benchmarks while reducing memory-induced degradation.

Key Contributions

• WikiTrace is an evidence-linked memory system that stores each lesson as an insight linked to a summary of its trajectory outcome and to the raw trajectory, allowing agents to check lessons against their original execution.
• It contributes a reassessment mechanism for procedural memory: on a new task, the agent starts from all insights and, when an insight lacks needed details, conflicts with another, or comes from a failed trajectory, descends through the links with terminal tools to recover missing details or revise the insight before acting.
• Evaluation across five agentic benchmarks spanning eight suites shows WikiTrace achieves the best performance on every suite, improves over a no-memory agent by 10.7 points on average, outperforms the same-insights baseline by 3.2 points, and reduces memory-induced regression from 6.5% to 4.5%, with larger gains on harder tasks where linked evidence is consulted more often and changes the plan.

Introduction

Large language model agents increasingly use external memory to convert past task trajectories into reusable lessons, improving multi-step tool-use and text-world performance without updating model parameters. However, memory does not always help: because a trajectory has only a single success or failure label, systems must guess which actions mattered, and existing methods often expose lessons without the source trajectories needed to verify them. This can cause memory-induced regression, where an agent fails tasks it would solve without memory. The authors introduce WikiTrace, which keeps each lesson linked to a compilation and the raw trajectory so the agent can recover missing details or revise misleading guidance before acting. Across five benchmarks and eight suites, WikiTrace improves over no-memory and insight-only baselines while reducing regression from 6.5% to 4.5%.

Dataset

The authors construct WikiTrace as a source-linked memory dataset with four components:

• Raw trajectories: Per-run records are preserved unchanged, including task metadata, each turn’s reasoning, action and observation, and whether the run succeeded.
• Compilations: A compiler processes each task, the agent’s trajectory, and a success/failure signal. It uses separate templates for successful and failed runs. Successful runs summarize decisions associated with success; failed runs record plausible explanations for failure. These are treated as interpretations of the run, not verified explanations.
• Insights: At most three insights are extracted per run. Each insight is a separate document stating what to do, when it applies, which run it came from, and whether that run succeeded. Insights are generated independently per run and are never updated or deleted, so the pool may contain redundant or conflicting insights.
• Source links and indexes: Each insight links to its source compilation and raw trajectory. This allows a test-time agent to trace where an insight came from and how the source run was interpreted. Each layer also has an index for navigation.

Processing details:

• Each run is processed independently so every extracted lesson remains connected to its source experience.
• Raw trajectories are kept unchanged.
• The compiler produces one compilation per run and up to three insights.
• The excerpt does not specify corpus size, task sources, training split, or mixture ratios.

Method

The authors introduce WikiTrace, a framework designed to mitigate memory-induced degradation in LLM agents by preserving a verifiable path from each extracted insight back to its source trajectory. The system operates in two primary phases: memory construction and test-time evidence access.

In the memory construction phase, the authors process a fixed corpus of past runs $\mathcal{D}_{\mathrm{train}} = \{(x_i, r_i, s_i)\}_{i=1}^N$Dtrain​={(xi​,ri​,si​)}i=1N​, where each run comprises a task $x_i$xi​, a trajectory $r_i$ri​, and a binary success outcome $s_i$si​. A compiler processes each run independently to generate a compilation $c_i$ci​ and a set of insights $I_i$Ii​, formulated as $(c_i, I_i) = \mathrm{Compiler}(x_i, r_i, s_i)$(ci​,Ii​)=Compiler(xi​,ri​,si​). The compilation summarizes the key decisions and supporting observations, using distinct templates for successful and failed runs. The compiler also extracts up to three insights per run, with each insight detailing the recommended action, its applicability conditions, and its source run. To ensure traceability, the raw trajectories are preserved unchanged, and every insight is explicitly linked to both its source compilation and the original raw trajectory. This creates a multi-layered, source-linked memory structure $\mathcal{M} = (R, C, I, E)$M=(R,C,I,E).

During the test-time phase, the agent leverages this structured memory to solve new tasks. Initially, the agent receives all extracted insights alongside the three past runs with tasks most similar to the current one. Rather than blindly following the insights, the agent is prompted to descend to the source runs when an insight lacks sufficient detail, originates from a failed run, conflicts with another insight, or when a subsequent action fails. The source-linked memory is mounted as read-only files, allowing the agent to use tools to search for specific evidence, such as command formats or contextual observations, before committing to an action. This descent mechanism enables the agent to verify the validity and applicability of an insight in real time, effectively recovering omitted observations and preventing the execution of erroneous or out-of-scope lessons.

Experiment

WikiTrace is evaluated against No Memory and existing memory methods such as AWM, ExpeL, and ReasoningBank, plus an Insights Only variant, across eight suites spanning simulated text worlds and tool-use environments. It consistently achieves the best performance, with gain and regression analyses showing that it improves previously failed tasks while reducing failures on previously solved ones, a benefit attributed to access to source runs. Further experiments indicate that descending into source evidence helps most on harder tasks and after blocked attempts, allowing the agent to revise or supplement insights, and that failed trajectories are useful but safer when kept with their source evidence. The approach also remains effective with larger baseline budgets and a different model, and it can resolve conflicting insights by revisiting raw trajectories.

WikiTrace achieves the highest performance on all eight suites, averaging 90.2 compared with 79.5 for the no-memory baseline. It exceeds the insights-only condition by 3.2 points on average, indicating that linked source evidence adds benefit beyond insights alone. WikiTrace is also the only memory method that matches or exceeds the no-memory baseline on every suite. WikiTrace leads all evaluated methods across every suite and on the overall average. Existing memory methods do not consistently match the no-memory baseline, while WikiTrace matches or exceeds it on all suites. Linking insights to source evidence yields a 3.2-point average gain over using insights alone and helps reduce memory-induced degradation.

WikiTrace shows larger score improvements over Insights Only when its evidence changes the agent's plan rather than leaving the plan unchanged. The biggest relative benefits come from introducing a new fact or revising an insight, while confirming the plan or not using the evidence yields the smallest gains. This indicates the value of WikiTrace is tied to evidence that actively reshapes the agent's decisions. Score gains over Insights Only are larger for plan-changing evidence than for plan-unchanging evidence. Among evidence effects, introducing a new fact leads to the largest improvement, followed by revising and refining insights. Confirming an existing plan or leaving evidence unused produces the smallest relative gains.

WikiTrace achieves the highest performance across all evaluated suites with GLM-5.3-flash, averaging 92.7 and surpassing insights-only memory by 1.8 points and no memory by 21.5 points. AWM, ReasoningBank, and ExpeL each fall below no memory on at least one suite. This pattern matches the DeepSeek results and supports the benefit of source-linked memory across models. WikiTrace leads all methods on PlanCraft, ScienceWorld, seen tasks, and unseen tasks. AWM, ReasoningBank, and ExpeL each underperform No Memory on at least one evaluated suite.

The experiments evaluate WikiTrace against no-memory and insights-only baselines as well as existing memory methods across eight suites and two language models. WikiTrace consistently achieves the highest overall performance and is the only method that matches or exceeds the no-memory baseline on every suite, while linking insights to source evidence provides consistent gains. These gains are largest when evidence introduces new facts or revises insights and changes the agent's plan, indicating that the value of source-linked memory comes from actively reshaping decisions. Replication with GLM-5.3-flash shows the same pattern, with WikiTrace outperforming all methods and existing memory methods sometimes falling below no memory, supporting cross-model benefit.