One-sentence Summary

Researchers from Beihang University and collaborating robot companies (Agilex, Franka, et al.) propose EVA-Client, an open-source unified framework that decouples robot integration, inference strategies, and transport middleware to support real-time deployment, debugging, and evaluation of manipulation policies, and automatically records training-ready rollout data from every run, thereby closing the real-robot iteration loop.

Key Contributions

• EVA-Client provides a component-decoupled architecture that separates robot backends, inference strategies, and transport middleware, making any combination of these layers run out-of-the-box and ensuring robot-agnostic and signal-source-agnostic deployment.
• Inspectable Deployment, Debug, Collect, and Evaluation workflows support execution modes from open-loop simulation and single-chunk stepping to continuous control, enabling reproducible and debuggable real-robot experiments.
• Every evaluation automatically records rollout data in the training-ready LeRobot format, along with comprehensive per-run logs and an interactive side-by-side viewer, so physical evaluations feed external training in a closed loop.

Introduction

Robot manipulation research has increasingly adopted generalist policies built on multimodal foundation models, and a mature training ecosystem now provides modular frameworks, shared data formats, and scalable recipes. Despite this progress, deploying these policies on physical hardware remains fragmented: real-robot execution typically relies on method-specific scripts that couple observation handling, action scheduling, latency compensation, and evaluation in ad-hoc ways, making deployments difficult to inspect, reuse, or compare across platforms. The authors present EVA-Client, a unified client that closes this deployment gap by providing a single, component-decoupled layer for inference, data collection, and evaluation. It decouples policy servers, robot backends, and inference strategies, turning deployment from a per-model script into a reproducible, inspectable process that automatically captures rollout data in a training-ready format, thereby completing the real-robot iteration loop and feeding back into external training.

Dataset

The dataset is a collection of teleoperated demonstration episodes generated through EVA-Client’s Collect mode. Each episode captures a human operator driving a robot via a leader-follower setup, with the client recording synchronized state-action pairs and camera feeds.

Dataset composition and sources

• Source: Human teleoperation recorded live on ROS1, ROS2, or ZMQ transports. A leader arm is moved by the operator, and the follower arm mirrors it exactly.
• Composition: One dataset is created per task. Every episode stores per-step observations (robot’s measured state) and actions (commanded movement), both available as joint angles and, via forward kinematics, end-effector poses.
• Multimodal content: Each step includes a columnar table entry with state and action vectors, plus one H.264 video per camera. Dataset metadata is written alongside the episodes.

Key details and processing

• Format: Episodes are saved in the LeRobot layout, matching the same format used for offline replay and training. The fixed-rate assumption is enforced by synthesizing per-frame timestamps at exact intervals; the original jittery capture time is preserved in a separate field.
• Quality control: Every finished episode undergoes an automatic frame-by-frame check for non-monotonic timestamps, missing or malformed camera frames, incorrect vector dimensions, non-finite values, and video–table length mismatches. Flagged episodes are quarantined: offending fields are zero-filled to keep the table regular, and the issue is recorded for review. Operators then replay each episode open-loop in the console, assigning a pass/fail verdict and optional free-text note, which is written without altering the trajectory.
• Recording mechanism: Capture is gated by an explicit activation step, so no accidental recording occurs. A background writer appends episodes to disk, ensuring live capture never stalls.

How the data is used

• The collected episodes serve directly as supervised training targets for policies. Because both state and action are stored, a recording is replayable for review and immediately usable for behavior cloning.
• The paper uses the dataset to train a driving policy; no explicit training split or mixture ratios are described in this section, but the data is prepared in a task-specific, training-ready format with automatic quality checks and human review.

Method

The authors present EVA-Client, a unified framework designed to bridge the gap between trained manipulation policies and physical robot execution. Positioned as a thin client, it mediates between various signal sources and the robot hardware, handling the complexities of real-time deployment, debugging, and data collection.

At a high level, the framework operates between signal sources, which include trained policy servers like openpi, StarVLA, GR00T, and Dream-Zero, as well as human teleoperation, and the robot execution stack. The client sends instructions and observations to model-based sources, receives action chunks, and issues commands to the robot over ROS 1/2 or ZMQ while reading back synchronized observations. Internally, the architecture is organized into five layers with narrow interfaces: the transport layer abstracts observation capture and command delivery; the robot description layer declares actuator groups and observation schemas; the policy client queries signal sources; the inference strategy manages chunk scheduling and smoothing; and the CLI/web layer drives the session state machine.

To support diverse hardware, the transport layer decouples control logic from communication protocols. The ROS1 and ROS2 backends buffer topics in bounded deques and align timestamps to produce synchronized observation frames, dropping stale messages to prevent mismatched data. The dataset backend replays episodes for open-loop testing, while the ZMQ backend enables development without physical robots. Robot support is declarative; a robot-description object lists actuator groups, cameras, and topic mappings, allowing new platforms to be integrated by simply writing a description class. The framework further decouples the observation state space, policy output space, and publication space, enabling policies to predict end-effector poses while commanding joint-controlled arms via a continuous inverse kinematics (IK) solver built on PyRoki. This solver minimizes weighted costs for end-effector pose, rest pose bias, and velocity under joint limits, ensuring smooth trajectories.

The operation model treats deployment as an iterative debugging activity, exposed through a single web console.

This console provides a persistent interface across Debug, Collect, and Evaluation modes, featuring a control panel, a 3D visualization scene, and synchronized camera streams. Within the Debug workflow, users can choose from several execution granularities. Open-loop simulation routes actions to the 3D visualization for safe validation. Real single-chunk stepping allows users to advance one action chunk at a time on the physical robot to localize failures. Single-chunk sim-to-real stepping interleaves simulation previews with physical execution, providing a human checkpoint. Continuous execution runs the full real-time deployment loop.

Real-time execution of chunk-based policies introduces challenges due to inference latency and overlapping action horizons. The inference strategy must determine when to request new chunks and how to blend overlapping predictions into a smooth command stream.

The authors implement several strategies to address this. Synchronous execution requests and executes a full chunk before querying the next, which is reproducible but causes pause-and-go motion. Asynchronous prefetch with linear-overlap blending runs inference and control in parallel. When a new chunk arrives, leading actions corresponding to the inference delay are discarded. The remaining actions are merged with the current buffer over an overlap window $L$L using a linear blend:

$$a _ {i} ^ { \text {blend}} = w _ {i} a _ {i} ^ { \text {old}} + \left(1 - w _ {i}\right) a _ {i} ^ { \text {new}}, \quad w _ {i} = 1 - \frac {i}{L - 1}$$aiblend​=wi​aiold​+(1−wi​)ainew​,wi​=1−L−1i​

where $i = 0, \ldots, L - 1$i=0,…,L−1. This gradually shifts weighting from the buffered trajectory to the new prediction. Temporal ensembling aggregates predictions in absolute time, computing the executed action as an exponentially weighted average of all predictions covering the current timestep:

$$\bar {a} _ {t} = \frac { \sum_ {(j, a) \in \mathcal {P} (t)} e ^ { - m \rho_ {t} (j)} a } { \sum_ {(j, a) \in \mathcal {P} (t)} e ^ { - m \rho_ {t} (j)} }$$aˉt​=∑(j,a)∈P(t)​e−mρt​(j)∑(j,a)∈P(t)​e−mρt​(j)a​

where $\rho _ { t } ( j )$ρt​(j) is the local position of the call, favoring earlier predictions. Real-Time Chunking (RTC) conditions generation on the previously committed action chunk, shifted forward by the latency, to reduce seams at the source.

The choice of inference strategy significantly impacts performance across tasks with different demands.

For high-dynamics tasks like table tennis, synchronous execution stalls the arm, preventing it from tracking fast objects, whereas asynchronous scheduling maintains the control loop and enables successful rallies. For long-horizon tasks like cloth folding, asynchronous execution ensures smooth, coherent trajectories.

Finally, the framework supports data collection through teleoperated recording. An operator drives the robot, and the client records synchronized pairs of measured state and commanded action. Episodes are written in the LeRobot format with per-step observations, actions, and video, undergoing automated quality control to flag non-monotonic timestamps or malformed data before manual review.

Experiment

EVA-Client formalizes evaluation as a core subsystem by organizing trials into scored scenes with explicit milestones, enabling reproducible comparisons across multiple checkpoints under identical conditions. The system records three parallel action streams (raw predictions, smoothed actions, and executed commands) to make the inference pipeline debuggable and attribute behavioral errors precisely. A read-only web viewer then provides side-by-side checkpoint comparisons and auditable, video-linked records, transforming subjective physical runs into structured, reproducible evidence.

The debugging workflow offers a progression of execution modes that balance safety and observability against realism. Starting from open-loop simulation with zero hardware risk, users can step through action chunks on the real robot, preview each chunk in simulation before committing to hardware, and finally run continuous real-time execution at full control rate. Open-loop simulation renders policy actions purely in a 3D viewer, allowing safe inspection of behavior without any risk to hardware. Segmented sim-to-real mode previews each action chunk in simulation and asks the user to confirm it before executing on the real robot, providing a human checkpoint between prediction and physical execution.

EVA-Client unifies five inference strategies behind a single interface, each differing in whether inference overlaps execution and how overlapping action chunks are combined. The lone synchronous strategy produces pause-and-go motion, while the four asynchronous strategies avoid blocking but require a smoothing method to resolve temporal misalignment between predicted chunks. Synchronous execution is the only non-async strategy and introduces pause-and-go motion. Asynchronous prefetch with linear overlap blends latency-trimmed chunks over their overlap window, enabling continuous robot motion. Temporal ensemble applies exponentially weighted ensembling, in contrast to the simpler chunk-replace approach of naive asynchronous. Real-Time Chunking (RTC) combines server-side conditioning with a final linear-overlap pass, the only strategy using server-side smoothing.

The debugging workflow provides a safe progression from open-loop simulation to segmented sim-to-real previews with human confirmation, culminating in full real-time execution. EVA-Client unifies five inference strategies behind a single interface: synchronous execution causes pause-and-go motion, while asynchronous strategies like linear overlap blending and temporal ensembling enable continuous motion, with Real-Time Chunking additionally using server-side conditioning. These designs collectively offer flexible deployment, safe testing, and smooth policy execution on real hardware.