One-sentence Summary

Researchers from Tsinghua University, AgiBot, and Infinigence AI propose collective experience learning (CEL), a paradigm realized through failure-guided experience retrieval fine-tuning (FERT) and a fault-tolerant asynchronous training system (FATS) that enables deployed robot fleets to share successful execution trajectories without additional expert corrections, achieving success rates of 93.3-99.2% across five real-world manipulation tasks and improving multi-stage phone packaging with 96 robots from 79.2% to 98.3%.

Key Contributions

• Introduces collective experience learning (CEL), a paradigm in which deployed robots share execution experience to improve a common policy after expert-supervised initialization, without collecting additional expert demonstrations or corrective actions.
• Presents failure-guided experience retrieval fine-tuning (FERT), which uses failure contexts to retrieve relevant successful trajectory segments from shared fleet experience and uses their actions as supervision, and a fault-tolerant asynchronous training system (FATS) that coordinates experience collection, policy optimization, and policy distribution despite robot interruptions.
• Evaluates on five real-world manipulation tasks, where resulting policies achieve 93.3-99.2% success over 120 trials per task on robots excluded from training data collection; larger fleets generally improve performance, and on multi-stage phone packaging training with 96 robots raises success from 79.2% to 98.3%.

Introduction

Robot foundation models can translate visual and language understanding into physical actions, but initial capabilities do not automatically yield reliable execution when object properties, contact conditions, or multi-stage errors vary across deployments. Prior fleet-learning approaches often require expert demonstrations or corrective interventions during deployment, and human action supervision becomes difficult to scale as the robot fleet grows. The authors introduce Collective Experience Learning (CEL), a paradigm in which robots pool their own deployment experience to improve a shared policy without continued expert action supervision. They realize CEL through Failure-guided Experience Retrieval fine-Tuning (FERT), which uses failed executions to retrieve relevant successful trajectories for policy updates, and a Fault-tolerant Asynchronous Training System (FATS) that coordinates distributed collection, inference, and optimization across the fleet.

Method

The authors introduce Collective Experience Learning (CEL), a paradigm where robots share experience generated during deployment to improve a common policy. This approach connects experience collection and shared policy improvement in a closed loop. As shown in the figure below, the requirement for expert action supervision decreases when moving from learning from human demonstrations to learning from expert corrections, and finally to learning from collective experience generated within the robot fleet.

To realize CEL, the authors propose Failure-guided Experience Retrieval Fine-Tuning (FERT). This algorithm updates the policy using experience generated during deployment without collecting additional expert demonstrations or corrective actions. FERT uses successful robot trajectories as action targets and failure contexts to prioritize relevant successful experience for supervised policy updates.

The figure below illustrates the four logical steps of the FERT process.

First, the system performs asynchronous experience collection. Robots execute autonomously, and each completed trajectory receives a binary task outcome. Successful trajectories are retained in a collective experience buffer to supply action targets, while failed trajectories serve as temporary retrieval queries. This separation allows the system to use failures to guide learning without imitating unsuccessful actions.

Second, experience encoding creates a representation capturing both the scene and the robot configuration. The authors combine visual features with proprioceptive state. A frozen visual encoder processes camera views to produce visual tokens, which are pooled and concatenated. The retrieval representation $z_t$zt​ combines normalized visual features $h_t$ht​ and normalized joint state $s_t$st​:

$$z_t = \left[ \frac{h_t}{\max(\|h_t\|_2, \epsilon)}; \frac{s_t}{\max(\|s_t\|_2, \epsilon)} \right]$$zt​=[max(∥ht​∥2​,ϵ)ht​​;max(∥st​∥2​,ϵ)st​​]

Embeddings of successful trajectories are cached, while only the failure context query clip is encoded for failed trajectories.

Third, failure-guided retrieval identifies relevant successful segments. Since a binary failure label does not precisely locate the onset of failure, the system uses a short observation sequence preceding the endpoint as a retrieval query. The system retrieves temporally aligned segments from the experience buffer using slope-constrained subsequence dynamic time warping. The distance matrix evaluates the framewise distance between failure query embeddings and successful trajectory embeddings. The algorithm finds the minimum-cost alignment, and the top candidates with the lowest costs are selected and expanded to form the final retrieved set.

Fourth, prioritized fine-tuning converts retrieved intervals into sampling priorities. The priority $w_i$wi​ of a successful action chunk $A_i$Ai​ is updated based on its overlap with retrieved intervals:

$$w_i^{(n)} = \rho w_i^{(n-1)} + \sum_{R \in R_n} \frac{|\text{idx}(A_i) \cap R|}{H}$$wi(n)​=ρwi(n−1)​+R∈Rn​∑​H∣idx(Ai​)∩R∣​

Action chunks are sampled from the buffer in proportion to their priorities:

$$P(i) = \frac{w_i}{\sum_{j \in \mathcal{D}^+} w_j}$$P(i)=∑j∈D+​wj​wi​​

The policy is fine-tuned using a negative log-likelihood objective:

$$\widehat{\mathcal{L}}_{\text{FERT}}(\theta; \mathcal{B}) = - \frac{1}{|\mathcal{B}|} \sum_{i \in \mathcal{B}} \log \pi_\theta(A_i \mid o_i, g_i)$$LFERT​(θ;B)=−∣B∣1​i∈B∑​logπθ​(Ai​∣oi​,gi​)

This directs learning towards behaviors related to the current difficulties of the policy.

To support this learning loop across distributed resources, the authors design the Fault-Tolerant Asynchronous Training System (FATS). FATS coordinates robot execution, cloud inference, and policy optimization. It separates the learning loop into environment workers operating robots, inference workers computing actions, and training workers updating the policy.

The system architecture consists of a data plane called SwarmLink and a control plane called HotSwarm. SwarmLink connects robot execution to cloud inference and training using distributed channels. It pools inference requests and streams trajectory fields for cloud-side assembly, allowing interaction traffic to take priority over bulk data transfer. HotSwarm manages the fleet membership, detecting failures and returning failed robots to their places. It implements tiered recovery where environment workers are restarted in place, inference workers reload the current policy, and training workers resume from checkpoints. This design ensures that a failed or paused robot lowers the experience arrival rate but does not halt the entire training process, enabling decoupled progress across the fleet.

Experiment

The experiments evaluate FERT and FATS as a concrete realization of continual experience learning on five real-world tabletop manipulation tasks with AgiBot G2 robots, using held-out robot instances to measure task success. They show that fleet-generated experience with binary task outcomes improves policy reliability without additional expert demonstrations or corrective actions, outperforming single-robot online reinforcement learning, offline fleet reinforcement learning, and human teleoperation baselines. Scaling results indicate that success generally increases as more robots contribute experience, while ablations confirm the importance of failure-guided retrieval, moderate policy synchronization, retrieved-segment extension, and priority retention. System experiments in a 96-robot run demonstrate that FATS sustains experience collection under local faults and communication delays, with local recovery and decoupled progress avoiding the larger collection losses of synchronous rounds or full restarts.

Robot interaction and training experience differ sharply in size, cadence, and timing requirements. Interaction messages are small and must be returned before the next robot step, while experience episodes are much larger, complete about every few minutes, and do not need immediate delivery. These differences motivate serving inference independently from experience transfer and reusing inference observations for cloud-side trajectory assembly. Interaction messages are frequent, small, and latency-sensitive, whereas experience episodes are much larger and can be streamed asynchronously for training. Cloud-side assembly reuses observation data already sent for inference, reducing the amount of data remaining to transfer after an episode ends.

HotSwarm prevents artifacts left by a failed worker from corrupting its replacement by checking worker identity and applying artifact-specific responses. Stale membership records are refused, late control messages are ignored, connections are rebuilt under a new epoch, pipeline data are dropped and received again, and queued transfers are replayed on the new connection. These measures keep replacements distinct from dead processes and protect training data and connection state. Stale membership records and late control messages are handled by refusal and ignoring, preventing peers from waiting on dead processes and preventing late failure reports from retiring replacements. Pipeline data is dropped and received again, and queued transfers are replayed on a new connection, so training does not mix actions with another worker's observations or run transfers on destroyed connections.

Across the five manipulation tasks, FERT reduces the average failure rate from 19.0% to 2.67%, corresponding to an 86.0% relative reduction. The largest relative failure reductions occur on Phone Charging and Phone Packaging, while Flower Arrangement shows the smallest reduction. These results show improved reliability over the multi-robot HG-DAgger starting checkpoints. Average failure rate falls from 19.0% to 2.67%, a relative reduction of 86.0% across tasks. Phone Charging and Phone Packaging show the largest relative failure reductions, while Flower Arrangement has the smallest reduction.

The experiments validate a split serving design where small latency-sensitive interaction messages are handled separately from larger asynchronous experience episodes, and cloud-side trajectory assembly reuses inference observations to reduce remaining transfer overhead. They also test failure recovery, showing that HotSwarm rejects stale membership and late control messages, drops and re-receives pipeline data, and replays queued transfers on new connections so replacements do not inherit artifacts. Across five manipulation tasks, FERT lowers the average failure rate from 19.0% to 2.67%, an 86.0% relative reduction, with Phone Charging and Phone Packaging improving most and Flower Arrangement improving least.