One-sentence Summary

The authors propose MJev-Spatial, a System 1 multimodal decision model that unifies spatial choice, numerical, and point tasks by scoring candidates with pretrained query and key projections in a single forward pass without added parameters; built on Qwen3.5-4B and 12.3M question-answer pairs, it attains an overall score of 65.64%, ranks first on seven of twelve benchmarks, and runs 2.5× faster than the fastest autoregressive baseline.

Key Contributions

• MJev-Spatial is introduced as a System 1 multimodal decision model that unifies option selection, numerical prediction, and spatial point prediction as candidate classification using the backbone's pretrained query and key projections without added parameters, requiring a single forward pass per decision and generating no answer tokens.
• The model uses coarse-to-fine classification for numerical values and image regions, with hierarchical refinement for precision and tree expansion for multi-point prediction; joint stage supervision with teacher forcing trains all stages in one forward pass, while length-aware global batching and cross-GPU load balancing improve training throughput by 30.8%.
• Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score of 65.64% among eight tested models, ranks first on seven of twelve benchmarks, outperforms its unmodified autoregressive base model on all twelve, and runs 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.

Introduction

Spatial intelligence requires agents to answer visual questions by selecting spatial relations, estimating numerical quantities, or identifying points in an image, often under real-time constraints. Most existing vision-language models handle these tasks through autoregressive text generation, which ties the spatial prediction to token formatting and boilerplate, adds unnecessary supervision, and imposes sequential decoding plus parsing overhead. The authors introduce MJev-Spatial, a System 1 style multimodal decision model that unifies these three output types as classification over finite candidate sets. It scores candidates using the backbone’s pretrained attention query and key projections, applies hierarchical refinement for numerical and point precision without extra prediction heads, and is trained on 12.3 million spatial QA examples. On twelve spatial benchmarks, MJev-Spatial attains the highest mean score among eight evaluated models and achieves substantial inference speedups over its autoregressive base model.

Method

The authors leverage a unified classification framework to handle diverse spatial tasks, including choice questions, numerical estimation, and point localization. Given ordered images $I$I and a spatial question $q$q, the model predicts an answer by selecting from a set of candidates $\mathcal{C}_t$Ct​ constructed by a deterministic task controller at each stage $t$t. A shared scorer assigns a logit $z_{t,i}$zt,i​ to each candidate, and the prediction selects the highest-scoring class:

$$p_{\theta}(i \mid I, q, h_{<t}, \mathcal{C}_t) = \frac{\exp z_{t,i}}{\sum_{j=1}^{K_t} \exp z_{t,j}}, \quad \hat{a}_t = \arg\max_i p_{\theta}(i \mid I, q, h_{<t}, \mathcal{C}_t)$$pθ​(i∣I,q,h<t​,Ct​)=∑j=1Kt​​expzt,j​expzt,i​​,a^t​=argimax​pθ​(i∣I,q,h<t​,Ct​)

Only legal candidates enter the softmax, with padded slots receiving negative infinity. Choice tasks end after a single decision. Numerical and point tasks utilize a coarse-to-fine refinement process. For numerical prediction, the first stage selects a coarse interval and the second selects a fine cell within it, decoding to the cell midpoint. Point prediction requires fine resolution and local visual detail, so it uses three $3 \times 3$3×3 region selections, re-encoding crops from the original image between stages to avoid compounding resampling errors.

To implement this framework, the authors adapt a pretrained Vision-Language Model, specifically Qwen3.5-4B. Each candidate is expressed as text and encoded together with the visual tokens, question, and preceding refinement history in a causal sequence. Candidate descriptions end with an option-end marker, and the complete list ends with a decision marker. The hidden states at these markers represent the candidates and the full decision context. To prevent the final MLP from shifting representations toward lexical prediction at the expense of semantic matching, the authors compute candidate scores using the final full-attention layer's pretrained query and key projections. The decision representation supplies the query $Q$Q, and each candidate representation supplies a key $K$K. The score for candidate $i$i is the mean scaled inner product across query heads:

$$z_{t,i} = \frac{1}{H} \sum_{h=0}^{H-1} \frac{Q_{t,h}^\top K_{t,i,g(h)}}{\sqrt{d}}$$zt,i​=H1​h=0∑H−1​d​Qt,h⊤​Kt,i,g(h)​​

This design reuses pretrained features and matching projections, giving the classifier a strong initialization without requiring an additional prediction head or new embeddings.

For the training process, the authors convert existing spatial annotations into 12.3 million training question-answer pairs covering multiple-choice, point localization, length estimation, angle estimation, and counting tasks. They employ joint stage supervision, where every supervised decision uses categorical cross-entropy over its current candidate set. For a question $q$q with $S_q$Sq​ stages, the loss is calculated as:

$$\mathcal{L} = \frac{1}{N_{\text{actual}}} \sum_q \frac{1}{S_q} \sum_{t=1}^{S_q} -\log p_{\theta}(a_{q,t}^* \mid I_q, q, h_{q,<t}^*, \mathcal{C}_{q,t})$$L=Nactual​1​q∑​Sq​1​t=1∑Sq​​−logpθ​(aq,t∗​∣Iq​,q,hq,<t∗​,Cq,t​)

This formulation allows all stage losses to be evaluated in a single language-backbone forward pass, with earlier decisions unable to attend to later choices or crops. To handle varying sequence lengths and computational costs efficiently, the authors sort expanded sequences by length to form similar-length microbatches. A longest-work-first assignment then distributes these microbatches across GPUs, accounting for padded sequence length and visual-token work to balance the load. The authors adopt full-parameter fine-tuning of the language backbone using AdamW, while keeping the vision encoder and merger frozen.

Experiment

The evaluation compares MJev-Spatial with seven autoregressive VLMs on spatial prediction quality and inference latency, then uses controlled ablations to validate candidate scoring, numerical discretization, visual refinement, multi-point decoding, and training scheduling. The main results show that MJev-Spatial achieves the best accuracy-latency tradeoff, ranking first on seven of twelve benchmarks while reducing query latency substantially. Ablations indicate that reusing pretrained native Q/K scoring is sufficient, crop re-encoding is critical for precise localization, a coarser numerical codebook with argmax decoding is effective, hierarchical decoding supports multi-point prediction with pruning, and global length grouping with cross-GPU balancing improves training throughput by reducing padding.

The framework treats choice, numerical, and point questions as candidate classification problems with a shared scorer. Numerical outputs share a scalar interface but use separate codebooks, with length and angle following coarse-to-fine stages and count using a two-stage integer code. Point prediction uses a three-level spatial hierarchy, while a probability tree extends the same point classification to multi-point counting. Choice questions are answered in one stage over the supplied options and decoded through the recorded option permutation. Length and angle use one or two stages with coarse roots and fine cells, allowing zero or overflow outcomes before fine decoding. Count prediction uses two stages to combine coarse groups and integers into an exact count over the supported range. Point prediction progresses through three levels of spatial cells and decodes to the center of the final cell. The probability tree adapts the point hierarchy to recover multiple targets in counting evaluations.

MJev-Spatial achieves the highest overall accuracy across the twelve spatial understanding benchmarks, ahead of the next best autoregressive VLM, and ranks first on seven of twelve tasks. Its strongest results span visual question answering and point localization, while selected benchmarks such as BLINK and VSI full are led by other models. The model also outperforms its autoregressive base model on every benchmark. MJev-Spatial attains the highest overall score among all compared models. It ranks first on seven benchmarks, including CV-Bench, Robo-VQ, MMSI, RefSpatial, Where2Place, Point-Bench, and ERQA. Qwen3-VL-8B-Instruct is the closest overall competitor, while Molmo2-ER leads on selected tasks such as BLINK and VSI full. Against its autoregressive base model, MJev-Spatial improves on all twelve benchmarks, with the largest reported gains in robotic spatial VQA and referring spatial understanding. MJev-Spatial also achieves the lowest mean query latency among compared models, yielding a significant speedup over autoregressive baselines.

Comparison of candidate scoring variants shows no single method dominates across spatial tasks. Native Q/K scoring achieves the highest RefSpatial accuracy and the lowest VST-300 MAE, while the linear classifier performs better on Where2Place and RoboSpatial pointing. Native Q/K is favored because it reuses pretrained projections and adds no new trainable parameters. Native Q/K scoring reaches the best RefSpatial accuracy and lowest VST-300 MAE among the compared variants. Linear classifier scoring outperforms native Q/K on Where2Place and RoboSpatial pointing, indicating task-dependent advantages. Native Q/K reuses existing pretrained projections, avoiding a new prediction head and additional trainable parameters.

This ablation compares finer and coarser fine-stage binning for numerical prediction. The coarser setting uses far fewer fine cells and substantially worsens quantization error, yet downstream prediction error improves slightly. This suggests the final predictor tolerates the coarser quantization well. The coarser fine-stage configuration keeps the same coarse intervals but greatly reduces total fine cells. Quantization MAE is substantially higher with the coarser bins, while prediction MAE is slightly lower.

Matched training with alternative visual inputs shows crop re-encoding as the strongest overall strategy, achieving the best RefSpatial, CV-Bench, BLINK val, and MMSI-Bench scores and tying for the best SAT real accuracy. Original-image context and replay variants have task-specific strengths but do not consistently match crop re-encoding on spatial precision. The results support retaining crop re-encoding for local visual evidence in point prediction. Crop re-encoding leads or ties on five of the seven reported metrics, including the highest RefSpatial accuracy. Original-image context performs best on RoboSpatial VQA, while KV replay performs best on Where2Place. ViT replay and KV replay fail to recover crop-level RefSpatial performance, and KV replay has the lowest RoboSpatial VQA accuracy among the visual inputs.

The framework unifies choice, numerical, and point questions as candidate classification with a shared scorer, using hierarchical codebooks for length, angle, count, and point localization plus a probability tree for multi-point counting. Across twelve spatial benchmarks, MJev-Spatial ranks first overall and on seven tasks, improves over its autoregressive base, and achieves the lowest mean query latency. Ablations show native Q/K scoring is preferred because it reuses pretrained projections, and coarser fine-stage bins raise quantization error but are tolerated by the final predictor. Crop re-encoding is the strongest visual input strategy for local spatial precision, outperforming original-image context and replay variants.