One-sentence Summary

Researchers at Project Moonshot: Local Frontier Intelligence show that frequent compaction lets frontier language models maintain short contexts with little to no accuracy loss on long-horizon and long-input tasks, that software-engineering performance does not collapse even when taking at most one action per context, and that these results motivate semantic state space models (SSSMs) while challenging the need for long-context language models, since later tokens cost up to $7 \mathrm { x } .$7x.

Key Contributions

• Frequent context compaction allows frontier open weight models to solve broad long-horizon and long-input tasks with short contexts while sacrificing little to no task accuracy.
• A single-action-per-context setup for long-horizon software engineering tasks shows performance does not collapse when carrying only a compact semantic state, outlining semantic state space models (SSSMs).
• End-of-conversation compaction and selective input reading with compaction often match or improve full-context baselines, supporting the conclusion that long contexts are not necessary for the tested workloads.

Introduction

The authors examine why long input contexts are costly: token generation latency, throughput, and memory footprint grow with context length, and cloud serving costs can be 3.6x to 7x lower with shorter contexts even though retail token pricing hides this. Despite these costs, contemporary language models have expanded context windows far beyond the length of many practical workloads, optimizing for larger possible inputs rather than efficiency. The authors challenge the standard justifications for long contexts, which stem from long-horizon conversational tasks and long-input tasks, and show that full context is not inherently required at every step. Their contribution is evidence that compaction and selective input reading can match or improve full-context baselines while keeping live context short, supporting the conclusion that long contexts are not necessary for many long-context tasks.

Dataset

Dataset Description: Long-Context Benchmarks

The authors organize their evaluation data into two benchmark groups based on where long context originates: long-horizon task benchmarks, where interaction histories grow as an agent acts, and long-input task benchmarks, where the source material can fill or exceed the context window.

Long-horizon task benchmarks

• SWE-bench Verified: 500 human-validated real GitHub issue-resolution tasks from popular repositories. The agent must produce a patch that passes the repository’s held-out test suite.
• SWE-bench Multilingual: 300 manually curated issue-resolution tasks from repositories across nine programming languages. It tests repository exploration, implementation, and validation across language ecosystems.
• Terminal-Bench 2.1: 89 real command-line tasks executed in sandboxed containers, scored by per-task verifiers.
• FRAMES: 824 challenging multi-hop questions requiring information from two to fifteen Wikipedia articles. The agent searches a fixed Wikipedia corpus, opens relevant articles incrementally, and submits a free-form answer evaluated by an LM judge.
• τ²-bench: 184-task golden set across retail, airline, and telecommunications. The agent converses with a simulated user while operating domain-specific tools under policy constraints. Success requires satisfying the user request and bringing the environment to the correct final state.
• BFCLv3: 200 long-context episodes for multi-turn tool use. Agents must retain information across user turns and schedule state-changing tool calls.

Long-input task benchmarks

• LongBench v2: 503 challenging multiple-choice questions with inputs ranging from 8K to approximately 2M words. Six task families cover single-document and multi-document QA, long in-context learning, long dialogue, code-repository understanding, and structured-data understanding.
• LOONG: 1,600 multi-document questions whose answers typically depend on multiple complete financial reports, legal cases, or academic papers in English or Chinese. Tasks include locating, comparison, clustering, and chain-of-reasoning, with evidence distributed across multiple documents.
• Artificial Analysis Long Context Reasoning (AA-LCR): 100 human-crafted questions requiring reasoning across real-world document sets averaging approximately 100K input tokens. Seven document categories include company and industry reports, government consultations, academic and legal documents, marketing materials, and survey reports.
• Needle in a Haystack (NIAH): target sentences inserted at controlled depths in long unrelated essay text. The 125-instance grid varies insertion depth and document length, with contexts from 16K to roughly 500K tokens.
• RULER: synthetic long-context suite with thirteen tasks: eight needle-retrieval variants, variable tracking, common- and frequent-word extraction, and two multi-hop QA tasks. All thirteen tasks are evaluated at 256K and 500K tokens and scored by recall of the required answer set.
• Counting-Stars: 32 marked numbers scattered throughout a long text. The complete public set has 192 questions, spanning English and Chinese, acquisition and reasoning variants, and documents from 4K to 128K tokens. The reasoning variant includes initially incorrect counts that are later corrected.
• BABILong: short facts from bAbI reasoning tasks embedded into book text. It includes 550 questions across all ten task types and five input lengths from 64K to 1M tokens. Answering requires locating relevant statements and then performing operations such as fact chaining, counting, negation, or set tracking.

Curation and processing details

• Curation varies by benchmark: SWE-bench Verified uses human validation, SWE-bench Multilingual uses manual curation, Terminal-Bench uses sandboxed execution and verifiers, FRAMES uses an LM judge, and several suites use fixed public or golden evaluation sets.
• Needle-style processing includes inserting facts at controlled depths in NIAH, constructing synthetic retrieval and reasoning tasks in RULER, scattering marked numbers in Counting-Stars, and embedding bAbI facts into book text in BABILong.
• The authors note that the needle-style benchmarks generally do not rely only on simple lexical overlap. They require aggregation, variable tracking, or reasoning over multiple retrieved facts.

Usage in the paper

• These benchmarks are used to assess the paper’s research questions RQ1 to RQ3.
• The section describes evaluation workloads only. No training split, training mixture ratio, benchmark-derived training set, or cropping strategy for model training is specified.

Method

The authors establish a foundational methodology to evaluate and manage context in agentic workflows, utilizing two closely related base harnesses: mini-swe-agent and mini-agent. Both harnesses implement a minimal ReAct loop where the task is supplied in an initial user message, the model issues one Bash action at a time, and each action along with its result is appended to the conversation. The mini-swe-agent harness is tailored for software engineering and terminal tasks, providing a task repository or container. The mini-agent harness generalizes this approach for other benchmarks by retaining the same agent loop and interfaces while utilizing benchmark-specific instructions and submission rules.

To address the challenges of long conversational contexts, the authors introduce three distinct context-management regimes that dictate how interaction history is presented to the model. The first regime is Full-History Accumulation, which serves as the baseline by appending every model action and environment observation to a single conversation without removing or replacing any history. The second regime is End-of-Conversation Compaction. In this approach, the model follows the full-history ReAct loop until the conversational context crosses a predefined compaction threshold. The model is then instructed to produce a bounded, task-relevant compaction of its state. A higher total context limit reserves space for this compaction turn, and the harness subsequently starts a fresh conversation containing the original task and the compacted state. The third regime is Single-Action Compaction, which maintains a compact semantic state with essentially no conversational history. Each conversation receives the task and current semantic state, performs exactly one environment action, observes the result, and updates the semantic state before discarding the conversation and starting a fresh one for the next action.

For tasks involving long inputs, the framework enables selective input reading by storing long inputs as documents in the agent's filesystem environment rather than placing their full contents in the initial conversation. The task message identifies the available documents, and the agent performs selective reading using standard Bash tools. This method explicitly makes selective input reading an environment action, effectively reducing a long-input workload to a long-horizon workload.

Building upon these observations, the authors propose the concept of a Semantic State Space Model. This model is post-trained to comprehend an input language description of a conversation state, reason about the appropriate next action, take the action, register its outcome, and update the language description of the state. Unlike traditional State-Space Models that maintain a black-box neural state between token generations, a Semantic State Space Model maintains an explainable natural language state between turns of a continuing conversation. This design allows separate inference instances to seamlessly continue the conversation toward completion.

The utility of this architecture lies in its increased token and data efficiency. By maintaining short contexts, the model avoids the computational and monetary penalties associated with long context windows.

As shown in the figure below:

The data illustrates the dollar cost of generating one million output tokens on an existing input KV cache as a function of the context window size. Serving models at a context window size of 256K rather than 16K incurs significantly higher costs per generated token across various models. This substantial cost benefit underscores the practical advantage of maintaining short contexts through mechanisms like Single-Action Compaction and Semantic State Space Models, making them highly efficient alternatives to full-history-accumulating models.

Experiment

The experiments assess whether long-horizon agentic tasks and long-input reasoning tasks can be solved without keeping long contexts in the model, using frequent context compaction and selective input reading across a broad set of benchmarks with closed-weight models such as Claude Opus 4.5 and GPT 5.6 Sol, plus complementary open-weight and latency/throughput measurements. For long-horizon workloads, compacting the agent history once it reaches restrictive thresholds broadly maintains task accuracy while enabling substantial estimated cost savings, although some tasks may still require selective access to past information. For long-input workloads, selective reading and compaction keep contexts short and generally match or improve baseline accuracy, with conversation length remaining roughly bounded as document size grows. The final single-action compaction experiment shows that even reducing each interaction to one action and a compact semantic state preserves or improves accuracy for Claude and causes only small reductions for GPT, supporting the conclusion that long contexts are not necessary for these tasks.

The comparison shows that well-known literary works range from tens of thousands to hundreds of thousands of tokens, while model context windows range from hundreds of thousands to one million tokens. Most shorter works fit within the smallest listed model contexts, but several longer texts exceed those limits and the longest work requires the largest context window. Token estimates are about one third larger than word counts because of the approximate conversion factor. Literary works span from a short novel to an epic novel, while model contexts range from hundreds of thousands to one million tokens. The longest work fits only the largest model context, while several longer works, including religious texts and long novels, exceed the smallest listed model contexts.

The best EoCC configuration improves accuracy over the full-history baseline on all listed benchmark and model combinations while greatly reducing maximum peak context. SAC further reduces peak context to the smallest values and largely retains accuracy, with Claude Opus 4.5 matching or exceeding the baseline and GPT 5.6 Sol showing small declines. This indicates that selective input reading and single-action compaction can maintain performance without full-history context. Best EoCC accuracy exceeds the mini baseline for every listed model and benchmark pair, while its maximum peak context is far below the mini baseline in all cases. SAC has the smallest median and maximum peak context lengths across all listed cases and keeps Opus 4.5 accuracy close to or above baseline, while GPT 5.6 Sol incurs only modest accuracy reductions.

The first comparison evaluates token lengths of well-known literary works against current model context windows, showing that most shorter texts fit within smaller contexts while the longest works require the largest windows and token estimates run about one third above word counts. The second experiment tests EoCC and SAC configurations across benchmarks and models, finding that the best EoCC setup improves accuracy over the full-history baseline while greatly reducing maximum peak context, and SAC further minimizes context with only small accuracy declines on one model. These results validate selective input reading and single-action compaction as effective ways to maintain performance without full-history context.