One-sentence Summary

TurboVLA, a vision-language-action model from Huazhong University of Science and Technology and Huawei Technologies, replaces the conventional $V \to L \to A$V→L→A pipeline with a direct $V + L \to A$V+L→A mapping through lightweight bidirectional vision-language interaction and a compact decoder, achieving 97.7% success on LIBERO with 0.2B parameters and 31.2 ms latency on an RTX 4090.

Key Contributions

• TurboVLA reformulates the conventional V→L→A pipeline as a direct V+L→A mapping, using lightweight bidirectional vision-language interaction instead of a large language model as the central perception-to-action bridge.
• TurboVLA independently encodes visual observations and language instructions, fuses them via cross-attention to build task-conditioned representations, and decodes continuous action chunks through a compact decoder, substantially reducing model size, inference latency, and memory consumption.
• On the LIBERO benchmark, TurboVLA achieves a 97.7% average success rate with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB VRAM on a consumer-grade RTX 4090, matching or outperforming substantially larger VLA policies.

Introduction

Vision-language-action (VLA) models for robotic manipulation typically rely on a large language model (LLM) at the center of the perception-action pipeline, converting visual observations into language-aligned tokens, combining them with task instructions, and then decoding actions. This LLM-centric design transfers broad semantic knowledge but creates a computational bottleneck: processing every control step through a billion-parameter model leads to high inference latency and memory usage, limiting deployment on resource-constrained platforms even when action decoding is parallelized. The authors observe that once a task instruction is given, execution-level control only needs the instruction to condition how visual evidence guides actions, not the full generative reasoning of an LLM. They introduce TurboVLA, a V+L→A paradigm that removes the large language backbone, instead using a lightweight text encoder (like BERT) and a compact cross-attention module to directly fuse vision and language into a control-oriented representation, then predicting continuous action chunks in a single forward pass. This yields real-time manipulation at over 30 Hz on a consumer GPU with 0.2B parameters, matching the success rates of much larger VLA systems.

Method

The authors introduce TurboVLA, a direct and simple $V + L \to A$V+L→A paradigm for execution-level language-conditioned manipulation. Unlike LLM-centric VLA models, this method does not route visual and textual inputs through a large language model before action prediction. Instead, it relies on compact modality-specific encoders, a direct vision-language interaction module, and an action chunk decoder.

As shown in the figure below:

To reduce the overhead of an LLM-centered execution pathway while retaining sufficient instruction understanding, the authors use compact modality-specific encoders. Execution-level instructions typically specify manipulation skills through objects, attributes, and spatial relations. Therefore, instructions are encoded with a lightweight encoder such as BERT, and visual observations are processed with a vision encoder. Given a task instruction $x$x, the text encoder extracts token-level instruction features:

$$Z^l = P_l(f_{\text{text}}(x)) \in \mathbb{R}^{N_l \times d},$$Zl=Pl​(ftext​(x))∈RNl​×d,

where $P_l$Pl​ projects the encoder output into the policy dimension and $N_l$Nl​ is the number of instruction tokens. The complete token sequence is retained so that objects, attributes, and spatial relations remain available for fine-grained visual conditioning.

For each camera observation $I_n^{(i)}$In(i)​, the image encoder extracts spatial visual features, which are projected and augmented with positional and camera-view embeddings:

$$Z_n^{v,(i)} = P_v(f_{\text{img}}(I_n^{(i)})) + E_{\text{pos}}^{(i)} + e_{\text{view}}^{(i)}, \quad Z_n^v = [Z_n^{v,(1)}; \dots; Z_n^{v,(K)}].$$Znv,(i)​=Pv​(fimg​(In(i)​))+Epos(i)​+eview(i)​,Znv​=[Znv,(1)​;…;Znv,(K)​].

Here, $E_{\text{pos}}^{(i)}$Epos(i)​ preserves within-view spatial structure and $e_{\text{view}}^{(i)}$eview(i)​ identifies the camera source. Concatenating the $K$K streams retains complementary cues from multiple viewpoints. The robot state is encoded separately as $Z_n^s = f_{\text{state}}(s_n) \in \mathbb{R}^{N_s \times d}$Zns​=fstate​(sn​)∈RNs​×d, where $f_{\text{state}}$fstate​ is a lightweight projection network. State features are introduced directly to the action decoder, keeping cross-modal interaction focused on task-conditioned scene understanding.

Independently encoded visual and textual features do not yet identify which visual content is relevant to the current instruction. The authors employ a simple yet efficient vision-language interaction module to directly exchange information between the two streams. Let $V_n^0 = Z_n^v$Vn0​=Znv​ and $L_n^0 = Z^l$Ln0​=Zl denote the initial visual and instruction features. The interaction module progressively updates both streams through $N$N bidirectional cross-modal layers:

$$(V_n^\ell, L_n^\ell) = \text{FusionLayer}_\ell(V_n^{\ell-1}, L_n^{\ell-1}), \quad \ell = 1, \dots, N.$$(Vnℓ​,Lnℓ​)=FusionLayerℓ​(Vnℓ−1​,Lnℓ−1​),ℓ=1,…,N.

Each layer consists of layer normalization, bidirectional cross-attention, and modality-specific feed-forward networks with residual connections. Visual-to-instruction attention injects scene context into the instruction stream, while instruction-to-visual attention conditions visual features on task semantics. After the final layer, the updated streams are concatenated as $Z_n^{vl} = [V_n^N; L_n^N]$Znvl​=[VnN​;LnN​].

Finally, the authors use an ACT-style lightweight transformer decoder to map the fused multimodal representation and robot-state features to a sequence of continuous actions:

$$\hat{\mathbf{A}}_n = D_\theta(Q_a, [Z_n^{\text{vl}}; Z_n^s]) \in \mathbb{R}^{H \times d_a},$$A^n​=Dθ​(Qa​,[Znvl​;Zns​])∈RH×da​,

where $Q_a = [q_1, \dots, q_H]$Qa​=[q1​,…,qH​] contains $H$H learnable action queries and $D_\theta$Dθ​ denotes the action chunk decoder. All action queries are decoded in parallel, allowing the policy to predict the complete $H$H-step action chunk in a single forward pass without action tokenization or sequential generation. The model is trained through behavior cloning on expert action chunks. Given a target sequence $\mathbf{A}_n^* = [a_{n,1}^*, \ldots, a_{n,H}^*]$An∗​=[an,1∗​,…,an,H∗​], the training objective is $\ell_1$ℓ1​ loss, and no auxiliary language-modeling objective is required.

Experiment

The experiments evaluate TurboVLA across single-arm, bimanual, and real-world manipulation benchmarks to test its direct vision-language-to-action pathway against LLM-centric VLA policies. Results demonstrate that TurboVLA matches or surpasses the manipulation success of large LLM-based methods while using a small fraction of parameters, lower latency, and minimal inference memory, confirming that strong execution-level control does not require an LLM at the core. Ablation studies show that semantic language conditioning and bidirectional vision-language interaction are critical, but can be realized with lightweight text encoders and moderate interaction depth, validating the efficiency and effectiveness of the V+L→A design.

TurboVLA achieves 97.7% average success on LIBERO, matching or exceeding large VLAs while using a fraction of the resources. It demonstrates that replacing the LLM-centered execution pathway with a direct vision-language-to-action design yields strong manipulation performance and substantial efficiency gains. TurboVLA attains 97.7% average success, surpassing π_0.5 by 0.8 points while using only ~6% of its parameters and reducing inference latency from 93.6 ms to 31.2 ms. It outperforms VLA-JEPA in success rate, while being over 3× faster and using roughly 7% of its parameters.

On the RoboTwin 2.0 clean setting, per-task methods exhibit modest success rates, with DP3 leading at 55.2%. All baselines fall far behind TurboVLA's reported 97.7%, underscoring the benchmark's challenge and the effectiveness of the direct vision-language-to-action paradigm. DP3 achieves the highest success (55.2%) among per-task methods while using only 0.3B parameters and 78.4 ms latency. Diffusion Policy has the lowest success (28.0%) and extreme latency (794.1 ms), showing that vanilla action diffusion struggles on this benchmark. Despite embodied pretraining and 3.2B parameters, π0 reaches only 46.4% success, underperforming the smaller DP3.

Removing language conditioning causes a severe performance drop, especially on goal-directed tasks, confirming that visual priors alone are insufficient when multiple behaviors fit the same scene. Replacing natural language instructions with a learned task-ID embedding recovers most of the performance but still falls short of the full semantic model, indicating that language provides information beyond simple task identity. The full model with semantic instructions achieves the highest success rates across all evaluated task types. Without language, average success rate falls from 97.7% to 70.8%, with the Goal task dropping to just 11.6%. Task-ID embeddings recover to 95.4% average success but remain 2.3 percentage points below semantic instructions, showing that natural language offers more than closed-set task identity.

Comparing instruction encoders shows that BERT yields the highest average success, while T5-Small achieves a nearly identical score with the fewest parameters. SigLIP-Base underperforms the others, especially on goal and long-horizon tasks, but all encoders keep average success above 95%, indicating the architecture is not tied to a specific text backbone. T5-Small, despite having the smallest model size, attains an average success rate only 0.6 percentage points below BERT, the best encoder. The performance gap between BERT and SigLIP is widest on the long-horizon task, where BERT leads by 5.2 percentage points.

Bidirectional interaction between vision and language achieves the highest average success rate of 97.7%, outperforming direct concatenation and asymmetric cross-attention variants. Allowing both vision to query language and language to query vision provides complementary information that boosts performance, especially on the Goal and Long tasks. Bidirectional interaction improves average success by 2.5 percentage points over direct concatenation (97.7% vs. 95.2%). Visual Queries Language reaches 100.0% on the Obj. task, but Bidirectional Interaction is the top performer on Spa., Goal, and Long, leading to the best overall average.

TurboVLA is evaluated on LIBERO and RoboTwin 2.0, where its direct vision-language-to-action design achieves 97.7% average success with substantial parameter and latency reductions compared to large LLM-based VLAs. Ablations show that natural language conditioning is essential, as visual-only and task-ID variants degrade significantly, and that bidirectional cross-attention between vision and language yields the best overall performance. The architecture remains robust to the choice of instruction encoder, with all tested encoders maintaining above 95% success.