One-sentence Summary

Inspired by human cognitive development, researchers from KAIST and AITRICS introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems with controlled color modulation that trains LVLMs on 3D-to-2D projection, viewpoint transformation, and structural combination, enabling them to significantly outperform baselines and generalize to real-world spatial tasks despite the dataset's synthetic nature.

Key Contributions

• A new perspective shifts from annotation-heavy real-scene supervision to foundational spatial skill learning through structured synthetic tasks.
• The work presents SpatialBlock-15k, a scalable synthetic dataset of block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination, enhanced with controlled color cues for anchor-based reasoning.
• LVLMs trained on SpatialBlock-15k via direct answering (SpatialBlock-direct) or reasoning-based prediction (SpatialBlock-reason) significantly improve spatial reasoning and generalize to real-world scenes, outperforming existing spatial specialists.

Introduction

Large vision-language models (LVLMs) excel at 2D image understanding but struggle with spatial intelligence, the ability to mentally reconstruct 3D structure from 2D views. This gap limits their use in robotics and autonomous driving, where robust spatial reasoning is essential. Prior attempts to address this either add specialized 3D modules to model architectures or design spatial-reasoning tasks, but both paths depend on real-scene datasets that require dense, costly, and often noisy geometric annotations from external modules. The authors break from that annotation-heavy paradigm by taking inspiration from human development, where block play builds foundational spatial skills. They introduce SpatialBlock-15k, a fully synthetic dataset of block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination, augmented with controlled color cues to promote anchor-based reasoning. Training LVLMs on this compact synthetic dataset significantly improves spatial reasoning and transfers effectively to real-world scenes, offering a scalable and clean alternative to real-scene supervision.

Dataset

The authors introduce SpatialBlock-15k, a fully synthetic dataset of 15,000 block-stacking problems designed to evaluate and improve spatial reasoning in large vision-language models (LVLMs). The dataset moves away from annotation-heavy real-scene 3D labels by procedurally generating tasks that probe core spatial abilities: spatial composition, mental simulation, and spatial integration.

Dataset composition and sources

• The dataset is entirely procedurally generated; it uses no real-world images or manual annotations.
• It consists of three question types, each containing 5,000 samples. Every sample incorporates color as a functional visual cue rather than just decorative variety.

Key details for each subset

• Q1: 3D-to-2D projection with depth cues – Color encodes depth ordering from a given viewpoint. The model must predict the 2D appearance, requiring it to reconstruct the 3D structure, perform mental rotation, and resolve occlusions using color as an explicit depth indicator.
• Q2: Viewpoint transformation with anchor blocks – One block is assigned a distinct color to serve as an anchor. The model must preserve the anchor’s structural role and the relative positions of all blocks under self-rotation or viewpoint changes.
• Q3: Structural combination with overlapping blocks – Two structures are shown as separate images; attachment locations are indicated by overlapping colored blocks (transparent regions). Color acts as a shared reference point, forcing the model to align same-colored blocks across images and infer the combined global structure.

How the paper uses the data

• The dataset provides a structured framework for both training and evaluating spatial reasoning in LVLMs. The paragraphs do not specify a train/test split; it is likely used as a training corpus for fine-tuning or as a benchmark to measure spatial intelligence improvements.

Processing and construction details

• All problems are built on block-stacking configurations that require understanding occluded blocks, simulating transformations, and integrating components.
• Color is applied according to task-specific rules: depth-based coloring for Q1, a single anchor color for Q2, and matching colors for overlapping regions in Q3. This design encourages models to use visual cues as reference points for relational reasoning, mirroring how humans use salient objects to interpret complex scenes.

Method

To foster spatial intelligence in Large Vision-Language Models (LVLMs), the authors propose a framework built upon block-stacking problems that require structural spatial understanding. As shown in the figure below, solving these problems demands core spatial abilities: (a) spatial composition to reconstruct complete 3D structures from visible configurations, (b) mental simulation to predict outcomes of transformations, and (c) spatial integration to anticipate results when combining multiple components.

Based on these abilities, the authors define three question types targeting distinct aspects of spatial reasoning, which are further extended with visual cues to simulate real-world complexity. Refer to the framework diagram for examples of these tasks, including 3D-to-2D projection, viewpoint transformation, and structural combination, both with and without color-based functional guidance for depth ordering and anchor-relative relationships.

To train LVLMs on these tasks, the authors propose two distinct training strategies targeting different aspects of model capabilities: direct answer prediction and reasoning-based prediction.

For the Direct Answer Prediction Model, the authors focus on cultivating rapid inference by mapping visual inputs directly to their corresponding answers. Given a textual query $q$q and image $v$v, the model parameters $\theta$θ are optimized to predict the next token $y_i$yi​ given the preceding context $y_{(1:i-1)}$y(1:i−1)​ by minimizing the standard cross-entropy loss:

$$\mathcal{L}_{\mathrm{ce}}(\theta) = - \sum_{i} \log P(y_i \mid y_{(1:i-1)}, q, v).$$Lce​(θ)=−i∑​logP(yi​∣y(1:i−1)​,q,v).

This strategy directly optimizes answer precision, establishing a foundation for high-fidelity spatial problem-solving.

For the Reasoning-based Prediction Model, the authors enable the model to articulate its internal logical paths. To ensure a solid foundation, they first initialize the model using Low-Rank Adaptation (LoRA) rather than full-parameter fine-tuning, which preserves the model's inherent Chain-of-Thought reasoning capabilities. This avoids the need for a cold-start phase using synthesized trajectories from larger teacher models, which often exhibit low accuracy on these spatial tasks.

Following LoRA-based initialization, the model is optimized using reinforcement learning via Group Relative Policy Optimization (GRPO). The authors design a multi-objective reward function to evaluate response correctness and reasoning trace quality:

$$\mathcal{R}(y, o) = \mathcal{R}_{\mathrm{acc}}(y, o) + \mathcal{R}_{\mathrm{format}}(o) + \mathcal{R}_{\mathrm{len}}(o),$$R(y,o)=Racc​(y,o)+Rformat​(o)+Rlen​(o),

where $o$o is the model-generated prediction for question $c = (q, v)$c=(q,v) with ground-truth answer $y$y. The accuracy reward $\mathcal{R}_{\mathrm{acc}}$Racc​ verifies if the final answer matches $y$y. The format reward $\mathcal{R}_{\mathrm{format}}$Rformat​ enforces the required structure with the final answer enclosed in specific tags. The length reward $\mathcal{R}_{\mathrm{len}}$Rlen​ ensures the response length $L$L satisfies $50 < L < 1024$50<L<1024. All components are implemented as binary rewards.

Given this reward, for each question $c$c, the old policy model $\pi_{\theta_{\mathrm{old}}}$πθold​​ samples a group of candidate responses $\{o_1, o_2, ..., o_G\}${o1​,o2​,...,oG​}. Each response $o_i$oi​ receives a reward $r_i = \mathcal{R}(y, o_i)$ri​=R(y,oi​), from which the advantage $A_i$Ai​ is computed. The model is then updated by maximizing the GRPO objective:

$$\mathcal{J}_{\mathrm{GRPO}}(\theta) = \mathbb{E}_{c, \{o_i\}} \left[ \frac{1}{G} \sum_{i=1}^{G} \min \left( \frac{\pi_{\theta}(o_i | c)}{\pi_{\theta_{\mathrm{old}}}(o_i | c)} A_i, \mathrm{clip} \left( \frac{\pi_{\theta}(o_i | c)}{\pi_{\theta_{\mathrm{old}}}(o_i | c)}, 1 \pm \varepsilon \right) A_i \right) - \beta \mathrm{KL}[\pi_{\theta} || \pi_{\mathrm{ref}}] \right],$$JGRPO​(θ)=Ec,{oi​}​[G1​i=1∑G​min(πθold​​(oi​∣c)πθ​(oi​∣c)​Ai​,clip(πθold​​(oi​∣c)πθ​(oi​∣c)​,1±ε)Ai​)−βKL[πθ​∣∣πref​]],

where $\varepsilon$ε and $\beta$β are hyper-parameters, and $\mathrm{KL}[\pi_{\theta} || \pi_{\mathrm{ref}}]$KL[πθ​∣∣πref​] is the KL divergence between the policy model and the reference model.

Experiment

The evaluation fine-tunes vision-language models on the synthetic SpatialBlock-15k dataset and tests them on both in-domain and real-scene spatial reasoning benchmarks. Direct models improve mental rotation and viewpoint tasks, while reasoning-enhanced variants better handle complex multi-image inference, all without degrading general visual understanding. Ablations reveal that the combination of complementary question types, color-based visual cues, and the block-stacking task design are essential for robust performance, and that this formulation yields more transferable spatial reasoning than traditional spatial question formats.

Training on only 15K synthetic block-stacking samples enables models to surpass existing spatial specialists on real-scene benchmarks, with the reasoning variant excelling at complex logical inference and the direct variant achieving the best open-source performance on mental simulation tasks. Proprietary models still struggle, with in-domain accuracy remaining below 47%, while block-stacking tasks prove more effective for out-of-domain generalization than conventional spatial question types. The approach also preserves general visual understanding, as MMMU scores remain stable. GPT-5 leads proprietary models in overall score, yet all proprietary models achieve less than 47% on the in-domain benchmark. SpatialBlock-direct models yield large gains on MindCube, with a 4B variant improving over its backbone by 25.1% and reaching 51.3%, the top open-source result. The reasoning-enhanced model outperforms SpaceR and Spatial-SSRL on MMSI-Bench using only 15K synthetic training samples. Block-stacking synthetic data generalizes better to out-of-domain spatial benchmarks than conventional relative direction and distance questions. Models trained without real-scene images maintain stable MMMU performance, indicating no degradation in general visual understanding.

Ablation experiments show that the block-stacking task design, training on all three complementary question types, color-based visual cues, and LoRA-based initialization each contribute to robust spatial reasoning. Removing any component degrades performance, with the largest drops occurring when using only a single question type or omitting visual cues. These findings highlight the importance of diverse spatial supervision, task-specific design, and effective model initialization for generalization. Combining all three question types yields the highest overall accuracy (41.2), while single-type variants fall to 38.8 and 38.0, indicating complementary spatial supervision. Removing color visual cues reduces MindCube accuracy by up to 7.9 percentage points for the direct model and 7.3 for the reasoning model, underscoring their role in complex scene understanding. The proposed block-stacking task design outperforms an alternative spatial QA dataset on out-of-domain benchmarks, confirming that task formulation drives generalization beyond in-domain data.

Training on Synthetic-Real improves in-domain accuracy but harms most out-of-domain benchmarks, while SpatialBlock preserves in-domain performance and strengthens generalization on three of four out-of-domain datasets. This indicates that block-stacking tasks yield more transferable spatial reasoning than conventional relative direction and distance questions. Synthetic-Real substantially raises in-domain accuracy but lowers performance on Relative Distance, MindCube, and MMMU compared to the baseline. SpatialBlock matches baseline in-domain accuracy and improves results on Relative Distance, MindCube, and MMMU, with a slight drop on Relative Direction. SpatialBlock achieves the highest overall out-of-domain accuracy among the three training conditions.

Using only 15K synthetic block-stacking samples, the approach surpasses existing spatial specialists on real-scene benchmarks, with the reasoning variant excelling at logical inference and the direct variant achieving top open-source mental simulation results. Ablations confirm that combining three complementary question types, color-based visual cues, and LoRA initialization is essential for robust spatial reasoning, and block-stacking tasks generalize far better to out-of-domain benchmarks than conventional relative-direction questions. While proprietary models still struggle below 47% in-domain accuracy, the method preserves general visual understanding, demonstrating that task-specific synthetic data can effectively transfer to real-world spatial reasoning.