HyperAIHyperAI

Command Palette

Search for a command to run...

TokenEconomizer: コーディングエージェントのための経験駆動型ブラックボックスコスト制御

概要

最先端のコーディングエージェントの進歩により、AI支援ソフトウェア工学におけるトークンコストは急激に増大している。そのトークン効率は通常、独立したタスクで評価されるが、現実の開発者は大規模プロジェクトの継続的な構築・更新・保守にエージェントを利用する。この乖離を踏まえ、我々は現実的なコーディングタスクの系列にわたってエージェントを評価したところ、エージェントは環境に依存して反復的に発生するトークンコストを示し、既存手法ではその削減が困難であることを見出した。これらのコストを最小化するため、我々はタスク系列にわたって環境特有のトークンコストを削減する外部コントローラであるTokenEconomizerを提案する。この削減は、(i) 注入すべき操作上のマイクロスキルの決定、(ii) 永続的に無関係なツールの剪定、(iii) 非生産的な失敗ループへの介入タイミングの決定によって実現される。Claude Code、Codex、GitHub Copilot CLIを最先端モデルと組み合わせた評価において、TokenEconomizerは平均トークンコストを18.4~30.9%削減し、タスク成功率は同等であった。この削減効果は、テストしたすべてのエージェント・モデル設定のすべてのリポジトリで維持され、逐次的なウェブ開発やセキュリティ監査などの多様なドメインにも一般化し、タスク構造に応じてコストは6.4~21.2%低下した。本研究は、適応的ブラックボックスコスト制御が、現実的な設定におけるコーディングエージェントのトークンコスト削減の有望な方向性であることを確立する。

One-sentence Summary

Researchers from UC Berkeley, Microsoft Research, and Microsoft Security propose TokenEconomizer, an external black-box controller that reduces coding agents' environment-specific token costs over sequences of realistic coding tasks by injecting operational micro-skills, pruning persistently irrelevant tools, and intervening in unproductive failure loops, achieving 18.4–30.9% average token-cost savings with comparable task success across Claude Code, Codex, and GitHub Copilot CLI paired with frontier models and yielding 6.4–21.2% savings in sequential web development and security auditing.

Key Contributions

  • The paper introduces TokenEconomizer, an external black-box controller that learns from completed task traces to reduce environment-specific token costs in coding agents by selecting operational micro-skills to inject, pruning persistently irrelevant tools, and intervening in unproductive failure loops.
  • The method formulates these choices as environment-specific meta-strategies applied through the agent's prompt, tool configuration, and event stream, with recovery paths that retire stale skills, restore pruned tools, and redirect uncertain cases instead of stopping them.
  • Results show that across Claude Code, Codex, and GitHub Copilot CLI paired with frontier models, TokenEconomizer lowers average token cost by 18.4% to 30.9% with comparable task success, with reductions in every tested repository and generalization to sequential web development and security auditing where costs fall by 6.4% to 21.2%.

Introduction

Frontier coding agents such as Claude Code, Codex, and GitHub Copilot CLI can complete long-horizon software engineering tasks, but they incur substantial and compounding token costs because persistent instructions and growing tool histories are charged repeatedly across turns. Prior cost-reduction methods mostly optimize individual tasks through fixed prompt compression, tool restriction, repository maps, or routing, but these approaches often require agent modifications, trade away task success, produce inconsistent savings across repositories, and overlook the reusable experience that accumulates across tasks in a shared codebase. The authors analyze default agent traces and identify three recurring sources of removable cost: operational knowledge that is repeatedly rediscovered, loaded tools that remain unused, and failed trajectories that consume disproportionate spend. To address these issues, they introduce TokenEconomizer, an external black-box controller that uses completed execution traces to learn environment-specific policies for activating micro-skills, pruning irrelevant tools, and intervening in unproductive failure loops, yielding consistent cost reductions while maintaining comparable task quality across multiple agents and benchmarks.

Method

The authors begin by profiling agent traces to identify three primary sources of removable cost: repeated operational work within a repository, a large number of loaded tools that remain unused, and failed trajectories that consume a disproportionate share of the total spend.

As shown in the figure below:

To address these inefficiencies, the authors introduce TokenEconomizer, an external controller that applies three repository-agnostic meta-strategies to learn repository-specific policies from completed traces. This controller operates without modifying the underlying agent, instead acting through the instruction prefix, loaded tools, and live event stream.

Refer to the framework diagram:

The system optimizes a chronological task stream τ1,…,τn\tau_1, \ldots, \tau_nτ1​,…,τn​ within an environment. Before a task τi\tau_iτi​ begins, the controller C\mathcal{C}C determines the active policies based on the task description and the accumulated repository state Si−1S_{i-1}Si−1​:

(ΔPi,Ui,Bi)=C(τi,Si−1).(\Delta P_i, \mathcal{U}_i, B_i) = \mathcal{C}(\tau_i, S_{i-1}).(ΔPi​,Ui​,Bi​)=C(τi​,Si−1​).

Here, ΔPi\Delta P_iΔPi​ represents the set of micro-skills activated for the task, Ui\mathcal{U}_iUi​ is the restricted set of loaded tools, and BiB_iBi​ is a turn budget for failure-loop intervention. After the agent executes the task to produce a trajectory ξi\xi_iξi​, the controller updates the repository state to SiS_iSi​.

Meta-strategy I: Experience-grounded Micro-skills This module synthesizes operational knowledge from past trajectories. The controller condenses completed traces into digests of operational evidence, such as command templates and error patterns. An LLM proposer drafts candidate micro-skills, each pairing an operational instruction with executable detectors. A scope detector checks if the skill's situation arose in a trajectory, while a need detector records if the task actually required the skill. The support of a skill fff at task iii is calculated as:

qi(f)=∑j<iγi−j(1[ξj confirms f]−λ1[ξj contradicts f]),q_i(f) = \sum_{j < i} \gamma^{i-j} \Big( \mathbf{1}[\xi_j \text{ confirms } f] - \lambda \mathbf{1}[\xi_j \text{ contradicts } f] \Big),qi​(f)=j<i∑​γi−j(1[ξj​ confirms f]−λ1[ξj​ contradicts f]),

where γ<1\gamma < 1γ<1 discounts older evidence and λ>1\lambda > 1λ>1 penalizes contradictions more heavily. A skill is activated only if it is confirmed across tasks with non-overlapping edited files and its need rate meets a threshold. If qi(f)<0q_i(f) < 0qi​(f)<0, the skill is retired.

Meta-strategy II: Tool Pruning To reduce the recurring cost of the tool interface, this module omits schemas unlikely to help. Core tools for shell execution and file manipulation are always retained. Other tools are removed if they remain uncalled for a fixed number of consecutive tasks or if an LLM judge deems them irrelevant to the current task description. To recover from mistaken exclusions, the controller periodically restores the full tool catalog for a re-audit; any tool called during this audit is permanently unmasked.

Meta-strategy III: Failure-loop Intervention This module derives a repository-specific turn budget to identify unproductive loops. The budget for task τi\tau_iτi​ is defined as:

Bi=β⋅Qp({∣ξj∣:j<i,yj=1}),B_i = \beta \cdot Q_p \big( \{ |\xi_j| : j < i, y_j = 1 \} \big),Bi​=β⋅Qp​({∣ξj​∣:j<i,yj​=1}),

where QpQ_pQp​ is the empirical ppp-th percentile of turn counts from successful prior tasks, and β>1\beta > 1β>1 is a margin. During execution, a rule-based check monitors for repeated failure signatures or if the trajectory exceeds BiB_iBi​. If triggered, an LLM judge reviews a digest of recent actions to distinguish a stuck loop from a difficult but progressing task. A confident stuck verdict prompts a redirection with a summary of recent actions. If the pattern recurs, the rollout is terminated. A forced stop acts as a backstop at κBi\kappa B_iκBi​ turns if strict stall conditions are met.

Experiment

The authors first profile 60 SWE-Bench tasks across six repositories and find that fixed strategies such as RTK, Serena, and RepoMap produce mixed repository-level cost changes, motivating adaptive learned control. They then evaluate TokenEconomizer across five agent model settings on 277 SWE-Bench tasks, plus Web-Bench and two security benchmarks, comparing it with default execution, reduced reasoning effort, and fixed strategies. TokenEconomizer consistently reduces cache-aware cost while largely maintaining or improving Pass@1, with savings accumulating after a few tasks and ablations attributing most gains to tool pruning and micro-skill injection. Generalization results show substantial savings on Web-Bench and CredData, smaller but consistent savings on ReposVul, indicating that benefits vary with reusable structure and cost-reduction headroom.

Fixed cost-control strategies such as RTK and RepoMap show inconsistent cost effects, sometimes raising cost while leaving pass rates close to default. Lower reasoning effort saves cost but consistently reduces Pass@1. TokenEconomizer provides consistent cost reductions across settings and repository cells while generally preserving or improving pass rates, and its variants define the pass-preserving Pareto frontier. Lower reasoning effort delivers large cost savings but reduces Pass@1 in every setting. RTK and RepoMap produce mixed cost changes, with each increasing cost in many repository setting cells. TokenEconomizer alone lowers cost in every repository setting cell while preserving or improving pass rates in most settings. TokenEconomizer variants, with or without RTK or RepoMap, cover all pass-preserving Pareto frontier points.

Across the reported SWE-Bench runs, the combined meta-strategy stack achieves substantial cost savings relative to default execution while keeping Pass@1 close to or slightly above baseline. Tool pruning and micro-skill injection provide the largest individual savings, and all individual variants maintain pass rates within a few points of default. Savings generalize to other task structures but scale with reusable procedure and available cost headroom. Combining micro-skill injection, tool pruning, and the failure-loop guard produces the largest SWE-Bench cost reduction while keeping Pass@1 at or slightly above default. Tool pruning and micro-skill injection are the largest single contributors, both delivering double-digit savings, while the failure-loop guard contributes smaller single-digit savings. Each individual meta-strategy keeps SWE-Bench Pass@1 within three points of default, so the cost reductions do not come from a large quality sacrifice. Adaptive control variants occupy the Pareto frontier points that maintain or improve default pass rate, whereas reduced reasoning effort trades larger savings for notable pass-rate losses. Savings generalize across Web-Bench and CredData but are much smaller on ReposVul, where shorter trajectories leave less removable cost.

The first set of experiments evaluates fixed cost-control methods, reduced reasoning effort, and TokenEconomizer, finding that only TokenEconomizer consistently lowers cost across settings while preserving or improving pass rates, whereas RTK and RepoMap show inconsistent cost effects and reduced reasoning effort regularly lowers Pass@1. The second set evaluates a combined meta-strategy stack on SWE-Bench and generalization tasks, showing that micro-skill injection, tool pruning, and a failure-loop guard together deliver substantial cost savings while keeping Pass@1 near or slightly above default, with tool pruning and micro-skill injection as the largest contributors. Adaptive control variants occupy the pass-preserving Pareto frontier, while reduced reasoning effort trades larger savings for notable pass-rate losses, and savings generalize to Web-Bench and CredData but are smaller on ReposVul due to shorter trajectories.


AIでAIを構築

アイデアからローンチまで — 無料のAIコーディング支援、すぐに使える環境、最高のGPU価格でAI開発を加速。

AI コーディング補助
すぐに使える GPU
最適な料金体系

HyperAI Newsletters

最新情報を購読する
北京時間 毎週月曜日の午前9時 に、その週の最新情報をメールでお届けします
メール配信サービスは MailChimp によって提供されています