One-sentence Summary

Researchers at the University of Illinois Urbana-Champaign, Georgia Institute of Technology, and heelo.dev introduce agent-native primitives and the HEELO framework, reusable LLM-equipped units composed into task-specific graphs with evidence-based verification and library revision, improving real-world task benchmarks including Terminal-Bench 4.0, SWE-bench Verified, ToolBench, and heelo.dev tasks.

Key Contributions

• Introduces agent-native primitives, reusable units that equip a resource with a resident LLM so the resource can be applied in context and communicate with other primitives, along with the HEELO framework, which composes primitives into task-specific graphs, verifies results against environment evidence, revises implicated primitives, and publishes validated primitives.
• Instantiates HEELO with eight primitive families and evaluates them on more than twenty benchmarks; for example, Dev-Primitives raise Terminal-Bench 4.0 resolution from 37.3% under Codex to 55.5%, Tool Primitives raise ToolBench pass rate from 55.8% to 75.1%, and Agent Primitives raise mean accuracy over eight math, code, and QA benchmarks from 58.8% to 75.3%.
• Demonstrates real-world deployment via heelo.dev and an MCP server on 46 real-world tasks, where Claude Opus 5.5 with Claude Code and GPT-6.1 Sol with Codex improve from 9 completed tasks each alone to 39 and 35 with HEELO, with 1.9× and 2.3× less time and 1.55× lower cost per completed task. Companion analyses show activation is bounded by task demand and benefits stem from applying resources inside primitives.

Introduction

Large language model agents aim to complete real-world tasks end to end by acting through tools, code repositories, proofs, documents, and other agents. Although harnesses such as Codex and Claude Code make these resources reachable, frontier agents still fall short on practical benchmarks, with failures often occurring at the last mile where a resource must be applied in context, composed with other steps, and revised from observed evidence. Prior work tends to keep resources passive while leaving the competence to apply them with a central caller or a monolithic agent, so that competence is rebuilt per task and becomes harder to verify as libraries grow. The authors address this by introducing agent-native primitives, reusable resources that carry a resident LLM to apply the resource and communicate with other primitives, and by proposing HEELO, a harness engineering framework that composes a task-specific graph from an evolving primitive library, revises only the primitives implicated by task evidence, and admits new or improved primitives through a separate validation path.

Method

The authors leverage a library of reusable, agent-native primitives to construct task-specific harnesses for real-world problems. A real-world task $q$q arrives with a context $c_q$cq​, encompassing its inputs, environment, constraints, and acceptance criteria. An agent acts on the task through a harness $H$H, which serves as the control program deciding which operations run, in what order, and when the execution stops. Rather than relying on a fixed harness for every task, the framework builds a task-specific harness from a library $\mathcal{L}$L of reusable units, aiming to maximize the expected score $R(q, H) = \mathbb{E}_{\tau}[R_q(\tau) \mid H]$R(q,H)=Eτ​[Rq​(τ)∣H].

The core building blocks of this system are agent-native primitives. An agent-native primitive $P_i$Pi​ equips a reusable resource $a_i$ai​ (such as a tool schema, code component, or proof method) with a resident LLM $\mathcal{M}$M and exposes an interface $d_i$di​. When invoked with a natural-language request $x_i$xi​ in a context $c_i$ci​, the resident LLM applies the resource and returns a result $r_i$ri​ along with messages $m_i$mi​ directed to other primitives:

This design allows the primitive to assess its own relevance before selection, judging whether it applies to a request and proposing how it would be used. It also performs carried validation, applying the resource's own validation mechanisms, such as tests or schemas, before returning a result.

The framework realizes the harness construction and execution through three primary mechanisms. First, Library Selection and Composition (COMPOSE) builds the harness. Since a library typically holds more primitives than a single task requires, COMPOSE finds candidates through their interfaces, reads their relevance assessments, and constructs a task-specific plan $\Pi_t = \text{COMPOSE}(q, \mathcal{C}_t; \mathcal{L})$Πt​=COMPOSE(q,Ct​;L). This plan defines an activation set $\mathcal{A}_t$At​ and a dependency graph $\mathcal{E}_t$Et​ dictating how results and messages flow between the activated primitives.

Second, Evidence-Based Execution and Improvement runs and revises the plan. In each round $t$t, the activated primitives act, and their results are integrated into the task state $S_t$St​. The system executes the plan in the task environment to gather evidence $o_t$ot​, such as build status or test outcomes. A verification step yields a verdict $v_t \in \{\text{PASS}, \text{FAIL}\}$vt​∈{PASS,FAIL} and structured feedback $\phi_t$ϕt​, which names observed failures, implicates specific primitives, and provides revision guidance. If the task fails, the context is enriched with this feedback, and COMPOSE revises the plan, changing only the implicated work:

This loop continues until the task passes or a revision budget is exhausted.

Third, Validation and Publishing improves the library between tasks. New or improved primitives, mined from artifacts or harvested from verified runs, enter the library only after passing a separate validation path:

The method is instantiated across eight primitive families, including Tool, Code, Math, Knowledge, Agent, Dev, Navigation, and Harness primitives, each tailored to specific real-world task domains. To evaluate this approach in live environments, the authors built the heelo.dev platform, which exposes third-party actions as agent-native primitives and orchestrates them through planning, routing, execution, and verification loops for tasks like content distribution, customer acquisition, and personal assistance.

Experiment

Across eight primitive families and a real-world platform, the experiments evaluate HEELO against task-specific baselines and ablations, showing consistent gains from applying resources inside their owning primitives, preserving structured communication channels, and activating or revising primitives according to task needs. The results indicate that the benefit comes from the primitives themselves, their carried fields, validation and diagnosis mechanisms, and task-conditioned composition rather than from extra inference or mere retrieval access. In the real-world heelo.dev evaluation, connecting Claude Code and Codex to HEELO enables account-based and service-based tasks that the agents cannot complete alone, while reducing time and cost per completed task.

On the six ToolBench splits, HEART with Qwen3-8B achieves the best average pass and win rates among reported systems, outperforming both commercial and fine-tuned references. Its lead is largest on the hardest split where multi-tool instructions require cross-API dependencies. Vicuna and Alpaca fail across all splits, whereas ToolLLaMA improves with DFSDT and retriever augmentation but remains below HEART. HEART leads all reported reference systems on average pass and win rates across the six ToolBench splits. The largest performance gap appears on the hardest split with multi-tool instructions and cross-API dependencies, where HEART posts its strongest win rate. ToolLLaMA with DFSDT and retriever is the strongest fine-tuned reference but still trails HEART on both average pass and win rate. Vicuna and Alpaca score zero pass and win rates on every split, highlighting the difficulty of the benchmark.

On NESTFUL nested call sequences, HEART leads reference models across function and parameter F1, partial and full sequence accuracy, and win rate under both one-shot and three-shot settings. Its full-sequence accuracy is higher than the strongest baseline, GPT-5.4, in both settings. Smaller fine-tuned function-calling models such as xLAM variants and Hammer2.0 remain much weaker on nested sequences, often near zero full-sequence accuracy. HEART is best on all five NESTFUL metrics in both one-shot and three-shot demonstrations. HEART's full-sequence accuracy reaches 0.44 one-shot and 0.47 three-shot, above GPT-5.4's 0.40 and 0.42. HEART's win rate is 0.75 one-shot and 0.79 three-shot, compared with 0.66 and 0.68 for GPT-5.4. Fine-tuned xLAM models show near zero full-sequence accuracy in one-shot, underscoring the difficulty of nested tool calls.

Across the reported reference models, success rates are highest in retail and lowest in telecom, and all models decline as the pass criterion requires more independent successful trials. GPT-4.1 leads in retail, while GPT-o4-mini is more competitive in airline and telecom and operates with much lower token use, cost, and latency. Pass rates fall from Pass^1 to Pass^4 in every reported model and domain, showing that stricter repeated-trial success requirements reduce reliability. GPT-o4-mini undercuts GPT-4.1 on tokens, cost, and latency while posting the strongest reported telecom pass rates, though GPT-4.1 remains strongest in retail.

The full HEART configuration performs best on both ToolBench and NESTFUL. Removing any single harness mechanism lowers results, with the largest decline occurring when ToolFace and its Tool Primitives are removed. Result validation is the next most critical component, while removing request decomposition or value resolution causes smaller but still notable drops. Removing ToolFace and Tool Primitives causes the steepest drop across all metrics, reducing ToolBench pass rate to about one fifth of the full configuration and nearly eliminating NESTFUL accuracy. Removing result validation is the second most damaging ablation, with larger losses than removing request decomposition or value resolution. The full configuration leads every column, and ablation of decomposition or resolution preserves moderate performance, confirming that all components contribute but some are more essential.

On a 522-repository construction benchmark, adding execution feedback and diagnosis improves delivery scores and lowers zero-score rates relative to single-shot and retrieval-only baselines. The strongest configuration, retrieval plus adaptation, achieves the highest delivery scores on small, large, and all task sets and the lowest zero-score rate, while increasing recorded-call cost above the one-attempt baselines. Large tasks remain substantially harder than small tasks across all configurations. Single-shot and retrieval-only one-attempt runs have the lowest overall delivery scores and the highest zero-score rates, near 57-59%. Retrieval plus adaptation leads on D1, D5, and overall delivery scores and cuts the zero-score rate to 17.8%. Adding diagnosis to feedback yields the second-lowest zero-score rate among the compared configurations. Recorded-call cost rises with added stages, from under $1 for one-attempt baselines to about$1forone−attemptbaselinestoabout1.83 for retrieval plus adaptation. Even the best configuration scores much lower on the largest tasks than on the smallest tasks, indicating a persistent difficulty gap.

The experiments evaluate HEART on the ToolBench and NESTFUL function-calling benchmarks, domain-specific reliability tests, ablations, and a repository construction benchmark. HEART with Qwen3-8B outperforms reference systems on average ToolBench pass and win rates, with the largest advantage on multi-tool cross-API tasks, and it also leads NESTFUL nested-call metrics under one-shot and three-shot settings. Ablations show that ToolFace with Tool Primitives and result validation are the most critical components, while domain tests reveal that repeated-trial pass criteria reduce reliability and model strengths vary by domain. In repository construction, adding execution feedback and retrieval plus adaptation substantially improves delivery scores and lowers zero-score rates at higher recorded-call cost.