One-sentence Summary

Researchers from Southeast University, Nanjing University, Microsoft Research, and Tsinghua University present Embodied.cpp, a portable C++ inference runtime that unifies vision-language-action and world-action models through a five-layer architecture enabling modular multi-rate execution, latency-first fused inference, and extensible I/O, achieving 100.0% and 91.0% closed-loop task success rates on HY-VLA and pi0.5 respectively, and reducing world-action model block memory from 312.2 MiB to 88.1 MiB on heterogeneous robots.

Key Contributions

• Embodied.cpp is a portable C++ inference runtime for embodied models that captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters.
• The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction.
• Evaluations on the HY-VLA and pi0.5 vision-language-action models achieve closed-loop task success rates of 100.0% and 91.0%, and a LingBot-VA Transformer block benchmark reduces memory footprint from 312.2 MiB to 88.1 MiB, demonstrating that Embodied.cpp improves deployment efficiency while maintaining high accuracy across diverse embodied architectures.

Introduction

Recent embodied AI models such as vision-language-action (VLA) systems and world-action models (WAMs) have grown increasingly capable but must run on heterogeneous, resource-constrained edge hardware to become practical robot-side systems. Prior inference runtimes, however, are built for request-response LLM/VLM serving and miss three essential requirements for closed-loop embodied control: multi-rate execution, latency-first batch-1 optimization, and extensible embodied interfaces that go beyond token I/O. The authors introduce Embodied.cpp, a portable C++ inference runtime whose five-layer architecture (input adapters, sequence builders, backbone execution, head plugins, and deployment adapters) directly targets these challenges by decoupling components with different refresh rates, providing fused small-batch execution across diverse backends, and exposing pluggable modules for custom operators and heterogeneous inputs/outputs, thereby supporting both VLA and WAM families under a unified, extensible framework.

Method

The authorsdesign Embodied.cpp to address the unique challenges of embodied deployment, which differ significantly from traditional LLM and VLM inference. Unlike conventional serving that assumes synchronous request-response paths, embodied models operate in closed-loop control with heterogeneous modules and strict latency requirements on diverse robot-side hardware. To overcome these hurdles, the authors establish three core design principles: modular multi-rate execution, latency-first fused execution, and extensible operator and I/O support.

The runtime architecture implements these principles through a unified yet extensible framework. On the input side, input adapters absorb both online sensor streams including cameras, force or tactile signals, and IMU data, as well as offline dataset samples, routing them through a single typed embodied interface. At the core, the system maintains a shared embodied-model execution zone capable of hosting VLA models, World-Action Models (WAMs), and future variants. This central zone preserves explicit interfaces for future prediction, action experts, and final action generation.

Beneath this model execution zone, three supporting subsystems directly address the core deployment challenges. First, modular multi-rate execution handles decoupled scheduling and runtime state, allowing different components like perception stacks and action heads to operate at their optimal refresh rates without forcing a single synchronous path. Second, batch-1 heterogeneous hardware execution prioritizes stable control performance, supporting graph replay, buffer reuse, operator fusion, and backend-specific dispatch to ensure low-latency deployment across CPUs, GPUs, NPUs, and other accelerators. Third, an embodied AI kernel warehouse collects reusable operators and model-specific kernels required by evolving architectures. Finally, output adapters bridge the runtime outputs to simulators like Isaac Sim and Gazebo, as well as real-world robot software stacks such as ROS and Apollo Cyber RT, ensuring the deployment boundary remains stable as model structures and hardware targets evolve.

Experiment

The evaluation demonstrates that Embodied.cpp can deploy distinct VLA models (HY-VLA and pi0.5) through a unified C++ runtime, with HY-VLA reaching a 100% success rate on its task while exhibiting higher latency due to its larger backbone and multi-view inputs, and pi0.5 achieving lower amortized step costs thanks to a lighter backbone and longer action chunking. A preliminary WAM microbenchmark on LingBot-VA shows that the quantized C++ Transformer block reduces weight memory from 312 MiB to 88 MiB while maintaining negligible output drift (MAE below 0.033 and cosine similarity above 0.9997), offering initial evidence that large memory savings are achievable for WAM components with minimal accuracy loss.

Embodied AI models split into vision-language-action (VLA) models that map perception directly to actions and world-action models (WAM) that explicitly incorporate future prediction. Architecturally, both families range from monolithic designs to increasingly modular systems that separate planning, world modeling, and action generation. Deployment evidence shows that these structural choices affect latency and memory, and that quantization can drastically reduce memory while preserving output fidelity. VLA models follow a perception-to-action path, whereas WAMs make future prediction an explicit part of online control, extending the runtime from simple action generation to stateful predictive orchestration. Modular designs are now prevalent: hierarchical VLA splits planning and control, asynchronous VLA decouples module speeds, and WAMs separate world modeling and action through staged, shared-backbone, or latent-compression patterns.

The comparison shows that existing inference runtimes either lack native support for embodied model families or are limited to a single model type. General-purpose runtimes can achieve edge and heterogeneous hardware deployment but do not natively support VLA or WAM models, while vla.cpp covers VLA models and robots but not WAMs. Embodied.cpp is the only runtime that natively supports both VLA and WAM execution, modular optimization, and all listed deployment capabilities. vla.cpp is the only prior system with native VLA and robot support, but it lacks WAM support and offers only partial modularity and simulator integration. llama.cpp and ONNX Runtime achieve edge and heterogeneous hardware deployment, yet they require substantial custom integration for embodied model families and provide no robot or simulator interfaces.

Both HY-VLA and pi0.5 run correctly through the same C++ deployment, preserving their task behavior. HY-VLA achieves a perfect success rate on the RoboTwin benchmark but incurs higher latency due to its larger Hunyuan-VL backbone, three-view input, and video-history path. pi0.5 trades a modest success drop for dramatically lower step and inference latency by using a lighter PaliGemma backbone and a longer action chunk. A heavier backbone and multi-view visual input raise inference latency and GPU memory, as seen with HY-VLA compared to pi0.5. Increasing the action chunk length, as pi0.5 does, sharply reduces amortized per-step latency while maintaining competitive success.

A single-block microbenchmark of LingBot-VA compares the original BF16 Transformer block against the Embodied.cpp Q4_K quantized version. The quantized block needs much less memory (88.1 MiB vs 312.2 MiB) with nearly identical latency (3.171 ms vs 3.236 ms), and output fidelity remains high with a mean absolute error below 0.033 and cosine similarity above 0.9997. These results demonstrate that aggressive weight quantization can drastically reduce memory usage while preserving block-level computation fidelity. The Q4_K quantized Transformer block achieves a 72% memory reduction (88.1 MiB compared to 312.2 MiB) while maintaining nearly identical inference latency. Output fidelity is preserved with a mean absolute error below 0.033 and a cosine similarity above 0.9997, indicating negligible drift.

Embodied AI models are increasingly modular, with hierarchical or asynchronous VLA designs and staged or shared-backbone WAMs; Embodied.cpp uniquely supports both families natively. Quantization dramatically reduces memory (72% in a Transformer block) while preserving output fidelity, and deployment tests reveal that larger backbones and multi-view inputs increase latency, whereas longer action chunks can amortize per-step cost. Overall, the results highlight the importance of runtime support for modular embodied models and the effectiveness of quantization for efficient deployment.