One-sentence Summary

Researchers from Peking University, BIGAI, et al. introduce EmbodiedCLBench, the first embodied-simulation benchmark for evaluating continual learning in self-evolving agents, which uses compositional generalization to test whether agents can recombine basic-task skills to solve unseen advanced tasks, and experiments across multiple agent harnesses and learning methods show that basic-task experience yields consistent but sample-inefficient improvements, offering resources for embodied continual learning.

Key Contributions

• EmbodiedCLBench is introduced as the first embodied simulation benchmark for evaluating continual learning in self-evolving agents, with compositional generalization organized into atomic actions, basic tasks, and advanced tasks.
• Experiments across multiple agent harnesses and learning methods show that experience from basic tasks consistently improves performance on unseen advanced tasks over zero-shot execution, and this improvement remains stable as task complexity increases.
• The benchmark also reveals that current agents do not scale these gains with additional learning samples, while the released code and dataset support further research on continual learning in embodied environments.

Introduction

Self-evolving agents need continual learning so they can accumulate experience and transfer skills across tasks, which is especially important for embodied intelligence operating in complex and unpredictable physical environments. Prior continual learning benchmarks mostly focus on non-embodied digital or symbolic settings, while existing embodied AI benchmarks evaluate perception, navigation, manipulation, and planning but do not specifically measure continual learning. This leaves a gap in assessing whether embodied agents can reuse prior experience for unseen tasks. The authors introduce EmbodiedCLBench, the first embodied simulation benchmark for evaluating continual learning in self-evolving agents, and they use compositional generalization as a principled way to structure tasks, evaluation protocols, and skill transfer from basic to advanced tasks.

Dataset

The authors build EmbodiedCLBench as a benchmark for evaluating compositional generalization in embodied agents. The construction has four stages: environment setup, task generation, graph construction, and evaluation protocol.

• Sources and platform

  • The benchmark uses TongSim, a household simulation platform, in its closed-source version.
  • TongSim provides interactive objects such as cups, plates, cabinets, lamps, and appliances.
  • It supports executable actions such as walk to, look at, pick up, put down, open, close, and switch on.
  • Task descriptions and action candidates are generated by Qwen3.7-Plus and manually verified for correctness.
  • Images paired with graph nodes are captured from TongSim states.

• Dataset composition

  • The data is organized around atomic actions, basic tasks, and advanced tasks.
  • Atomic actions are single object interactions, for example picking up a cup.
  • Basic tasks combine a small number of atomic actions into practical household operations.
  • Advanced tasks combine more atomic actions than basic tasks.
  • The provided excerpt does not specify exact task counts or dataset sizes; the design emphasizes combinatorial object-action diversity.

• Graph construction and processing

  • Each graph node represents a unique environment state and is paired with an image and a set of action candidates.
  • Each executable action candidate corresponds to an edge that deterministically transitions to the next state.
  • For each task, the initial and goal states are defined, and breadth-first search is used to find the shortest path.
  • Distractor paths are created by expanding alternative executable actions that deviate from the shortest path.
  • Action candidates are classified into optimal actions, suboptimal but executable actions, and invalid actions that cannot be executed in the current state.

• How the data is used

  • Evaluation is split into a learning phase and a testing phase.
  • In the learning phase, agents complete basic tasks to accumulate experience.
  • In the testing phase, agents attempt unseen advanced tasks.
  • Basic tasks serve as memory and test-time learning data for self-evolving agents.
  • The evaluation is interactive and multi-turn: the agent receives a task description and an initial image, selects an action, transitions to a new state, and receives a new image with updated candidates.
  • Evaluation continues until the agent reaches the goal or exhausts the step budget.
  • The graph-based setup enables offline, deterministic, and reproducible evaluation without running the simulator.

Method

The authors propose a structured pipeline for constructing the EmbodiedCLBench, which comprises four sequential stages: environment setup, task generation, graph construction, and evaluation protocol.

The overall construction pipeline is illustrated in the framework diagram below.

In the task generation phase, the authors operationalize continual learning evaluation through compositional generalization. The system defines atomic actions as single interactions with objects, such as picking up an item. These atomic actions are then composed into two levels of tasks. Basic tasks combine a small number of atomic actions into simple household operations. Advanced tasks integrate elements from multiple basic tasks to form more complex, unseen compositional tasks. This hierarchical design allows the evaluation of whether an agent can recombine previously mastered skills to solve novel problems without learning from scratch.

Following task generation, the authors implement a graph construction module to enable offline and reproducible evaluation. Since the underlying simulation environment is closed-source, online simulation is impractical. Instead, the task procedure is modeled as a state-transition graph. Each node in the graph represents a unique environment state, paired with a first-person-view image and a set of action candidates. Each executable candidate corresponds to a directed edge that deterministically transitions the agent to the next state. To construct the graph, the system defines the initial and goal states for a given task and employs a breadth-first search algorithm to identify the shortest path between them. Furthermore, distractor paths are generated by expanding alternative executable actions from nodes along the shortest path, creating executable but suboptimal trajectories for robust evaluation.

During the evaluation protocol, the agent observes the current state and selects an action from the provided candidates. Based on the selection, the agent transitions to the corresponding next node in the pre-computed graph. This graph-based approach allows for parallel offline evaluation, flexible control over task complexity via graph branching, and fine-grained behavioral analysis by checking each decision against the annotated trajectories.

Experiment

The benchmark evaluates continual learning through compositional generalization in simulated household tasks, comparing zero-shot agents with agents that learn from basic task trajectories or from concise skill summaries before attempting unseen advanced tasks. Results show that prior experience consistently transfers from simple to advanced tasks, substantially improving success over weak zero-shot performance. Further analyses indicate that this benefit remains stable as compositional complexity increases, arises mostly after a single learning sample rather than scaling with more samples, shifts decisions toward optimal actions and away from invalid actions, and persists across different step budgets.

Agents achieve moderate zero-shot success on basic tasks, but zero-shot performance on advanced tasks is much lower. In-context learning and skill learning both improve advanced task success across harnesses and models, indicating that experience from basic tasks transfers to unseen advanced tasks. Error analysis shows this improvement is accompanied by fewer invalid actions and more optimal actions. Zero-shot success is moderate on basic tasks but drops sharply on advanced tasks. In-context learning and skill learning both improve advanced task performance across different harnesses and models, with neither method consistently outperforming the other.

Experiments evaluate agents on basic and advanced tasks across multiple harnesses and models. Agents achieve moderate zero-shot success on basic tasks but much lower performance on advanced tasks. In-context learning and skill learning both improve advanced task success, indicating that experience from basic tasks transfers to unseen advanced tasks, and error analysis shows these gains are accompanied by fewer invalid actions and more optimal actions. Neither method consistently outperforms the other.