One-sentence Summary

The Light Origins Team presents LightNav-0, a compact generalist embodied navigation model that, via a unified token interface with dual-channel pointing and a residual vector-quantized action tokenizer combined with temporally aware visual history compression, ER midtraining, supervised fine-tuning, and reinforcement learning, supports instruction following, open-vocabulary object navigation, and visual tracking, achieving state-of-the-art monocular success rates across all 10 public navigation simulation settings and zero-shot generalization to real-world robots, diverse scenes, and static and dynamic targets.

Key Contributions

• LightNav-0 is a compact generalist navigation model that uses a dual-channel pointing prefix for task-agnostic spatial intent and a residual vector-quantized action tokenizer, eliminating task-specific prediction heads.
• A training recipe combining temporally aware visual history compression, embodied-reasoning midtraining, supervised fine-tuning, and reinforcement learning enables a single monocular RGB policy to handle instruction following, open-vocabulary object navigation, and visual tracking across 2K+ scenes and 4K+ hours of data.
• LightNav-ER attains the highest complete-set average on 8 embodied-reasoning benchmarks, and LightNav-0 achieves state-of-the-art monocular success rates across all 10 public navigation simulation settings while transferring zero-shot to four real-world robot embodiments.

Introduction

Embodied navigation tasks such as following language instructions, searching for objects, and tracking moving targets demand agents that can ground goals in observations, maintain spatial context, and act across diverse environments and robot platforms. Most existing systems are designed for a single benchmark, relying on task-specific components like waypoint predictors, topological maps, or separate action heads. This fragmentation prevents open-vocabulary transfer and isolates navigation from the scaling benefits of modern vision-language models.

The authors propose a different design principle: a compact vision-language model can serve as a shared reasoning backbone for general navigation. They introduce LightNav-0, which retains a pretrained VLM without adding task-specific prediction heads. The model expresses spatial intent through dual-channel pointing—predicting an affordance point for feasible motion and an object point for the goal—as a unified interface across tasks and embodiments. This grounded spatial reasoning is combined with temporally aware visual compression and a residual vector-quantized action tokenizer that generates precise continuous trajectories from the same language-model head. Trained on a large corpus spanning instruction following, object navigation, and visual tracking, LightNav-0 achieves strong monocular performance across multiple simulation settings and transfers directly to real-world robots without task- or robot-specific adaptation.

Dataset

The authors construct the training data around two coupled mixtures, with a navigation corpus spanning over 2,000 scenes and 4,000+ hours of embodied trajectories. Embodied-reasoning (ER) mid-training draws from 36 sources to strengthen spatial understanding, temporal reasoning, and visual grounding. Supervised finetuning (SFT) then combines 16 navigation sources with 33 auxiliary ER and VQA sources. Under active sampling, 77.6% of optimization samples carry navigation-action supervision and 22.4% rehearse ER capabilities; these ratios describe the task-balanced optimization mixture, not unique corpus coverage.

Embodied Reasoning Data

• ER mid-training allocates sampling mass across four capability groups:
  • Pointing and grounding (35.14%): 13 sources (RefSpatial-2D/3D, PixMo Points, COCO Pointing, CoSyn Point, RefL4, RoboRefIt, RoboPoint, RoboAfford, HANDAL, FSD Free-Point, FSD Visual-Trace) covering referred-object localization, free-space selection, interaction affordances, and visual trajectory traces. These teach the VLM to express spatial decisions in the image plane; during navigation SFT the same capability is instantiated as the grid used by affordance-point and object-point tokens.
  • Single-image VQA (25.05%): 11 sources (SenseNova-SI Spatial, CLEVR Spatial VQA, SAT Spatial VQA, VSTP-SI Depth Comparison/Distance/Scene Caption/Measurement, VSTP-MI Correspondence/Object–Object Relation/Camera Motion/Scene Caption) that cover relative position, metric depth, object relations, camera motion, measurement, and affordance-oriented reasoning.
  • Video reasoning (19.81%): 9 sources (RoboVQA-Reasoning/Understanding, Robo-FAC Failure-VQA, VSI-590K Spatial, SIMS-VSI Spatial, ViCA-322K, LLaVA-Video-VQA, SQA3D-Situated, SpatialLadder Spatial) with clips up to 64 frames, providing supervision for temporal ordering, trajectory-aware spatial relations, planning, affordance prediction, future-state reasoning, and failure understanding.
  • Abstract embodied reasoning (20.00%): 3 sources (LLaVA-OneVision Spatial VQA, Euclid30K-Math, MMIF-23K-Instruct) that preserve broad linguistic and visual breadth and isolate frame-of-reference and multistep composition from natural-image appearance biases.
• Multi-image and ego–exo correspondence samples are drawn from both image-VQA and video pools, requiring the model to associate objects across viewpoints, reconcile egocentric and exocentric observations, and estimate camera motion.

Navigation Data

• Organized by task objective (instruction following, object-goal navigation, embodied visual tracking) rather than robot platform.
• Camera randomization is applied during generation: field of view sampled from [90°,130°], camera height from [0.5,1.5] m, and pitch from [−15°,15°].
• All samples are converted to a unified output sequence: an affordance point, an object point, and three residual vector-quantized trajectory tokens, allowing the three tasks to be interleaved in a single autoregressive stream.
• Instruction following: randomized-camera expert trajectories from R2R and RxR, self-distilled routes, and ScaleVLN data.
• Object-goal navigation: semantic and expert demonstrations from PIRL-Nav, HM3D, MP3D, VLNVerse, Habitat-GS, and InteriorGS; self-collected exploration trajectories from HM3D-OVON and MP3D; in-loop DAgger samples.
• Embodied visual tracking: randomized-camera person-following trajectories from EVT-Bench.
• Quality control: stop supervision is kept only at the final frame when the target is visible; episodes where the target is never observed are removed. Stop samples are capped at 2% of the mixture. Dual-channel pointing labels use a shared grid format, and each 10-waypoint action chunk is encoded by three K=256 RVQ levels. Trajectory clusters are balanced with a maximum cluster share of 5%.

INSIGHT-Bench

• A heterogeneous benchmark with 1,683 training scenes and 53,090 episodes, plus 210 evaluation scenes and 1,097 episodes, spanning mesh-based and 3D Gaussian-splatting environments.
• Pre-annotation: unlabeled Habitat-GS captures, HM3D/MP3D, and VLNVerse use an automatic multi-view pipeline; InteriorGS supplies official ground-truth boxes. Molmo2 performs open-set pointing on rendered views, metric depth lifts predictions to 3D, and a cross-view consistency gate retains an instance only when supported by at least two observations from distinct viewpoints with localization spread below 0.6 m.
• Data generation: a target and starting pose are sampled such that the target is visible in at least one offline inventory view (not guaranteed in the policy’s initial observation). A route is planned on the navigation mesh, the stopping location is anchored, and egocentric video frames with action labels are rendered.
• Instruction labeling: a rule-based route uses the detected camera view and goal-pixel horizontal position to determine direction wording in view-aware sectors. A video-conditioned route submits 6 evenly spaced frames and a trajectory summary to Seed2.0 for route-level language. Both routes produce drafts, which a rewriter broadens lexically while preserving the head noun, implied direction, and never introducing the opposite direction. A visual arrival check confirms the target appears in at least one of the last rendered frames.
• Diagnostic taxonomy: episodes carry scene-level labels (apartments, houses, commercial, institutions, outdoor) and instruction labels (base, direction, relation, extremum, ordinal), enabling fine-grained analysis of success patterns. Training and evaluation scenes are strictly disjoint; the training split averages 31.5 episodes per scene, the evaluation split 5.2.

Evaluation Benchmarks

• The ER checkpoint is evaluated on 8 benchmarks: Point-Bench, RefSpatial, RoboSpatial (POI and VQA tracks), Where2Place, CV-Bench, ERQA, and Emb-Spatial.
• Navigation evaluation uses 10 simulation settings across instruction following (R2R, RxR val-unseen), closed-vocabulary ObjectNav (MP3D, HM3D v1/v2), open-vocabulary ObjectNav (HM3D-OVON), and embodied visual tracking (STT and DT settings of EVT-Bench). All navigation benchmarks use the same checkpoint without task-specific fine-tuning.

Method

The authors formulate heterogeneous embodied navigation tasks as conditional token generation. At each decision step, the model receives a natural-language instruction and an egocentric RGB history, generating a dual-channel pointing prefix followed by a short-horizon action sequence. The decoded action is a trajectory of 10 future SE(2) waypoints, providing a common geometric interface to low-level controllers.

The model is instantiated from a pretrained VLM backbone comprising a native-resolution vision transformer and a 36-layer language model. Rather than introducing navigation-specific modules, the authors retain the pretrained architecture and augment only its vocabulary with indexed pointing and residual vector-quantized action tokens. Both intermediate spatial predictions and action codes are decoded through the original autoregressive language-model head. Tokens from the timestamped visual history, current observation, and instruction are interleaved within a single causal sequence.

To handle long-horizon context without unbounded visual-token growth, the authors compress history according to temporal recency. For a historical frame acquired at time $t_i$ti​, its age is defined as $\Delta T_i = t - t_i$ΔTi​=t−ti​. The sampling rate decays exponentially:

$$f_s(i) = f_s^{\mathrm{max}} \exp \left(- \frac{\Delta T_i}{\tau_s}\right)$$fs​(i)=fsmax​exp(−τs​ΔTi​​)

The selected frames are encoded independently. Given the resulting patch grid $\mathbf{V}_i$Vi​, the spatial pooling stride is set to:

$$s_i = \max \left\{1, \left\lfloor \exp \left(\frac{\Delta T_i}{\tau_p}\right) \right\rfloor \right\}$$si​=max{1,⌊exp(τp​ΔTi​​)⌋} $$\widetilde{\mathbf{V}}_i = \mathcal{G}_{s_i}(\mathbf{V}_i)$$Vi​=Gsi​​(Vi​)

where $\mathcal{G}_{s_i}$Gsi​​ denotes grid pooling. Temporally distant observations contribute fewer and coarser tokens, while the current observation retains the finest visual detail.

To bridge the gap between 2D token lattices and metric space actions, the authors encode each projected point as a channel-specific image-grid token. The current view is partitioned into $H_g$Hg​ rows and $W_g$Wg​ columns. A projected point $\mathbf{p} = (u, v)$p=(u,v) is assigned a flattened grid index:

$$r(\mathbf{p}) = \min \{H_g - 1, \lfloor H_g v \rfloor \}$$r(p)=min{Hg​−1,⌊Hg​v⌋} $$c(\mathbf{p}) = \min \{W_g - 1, \lfloor W_g u \rfloor \}$$c(p)=min{Wg​−1,⌊Wg​u⌋} $$i(\mathbf{p}) = r(\mathbf{p}) W_g + c(\mathbf{p})$$i(p)=r(p)Wg​+c(p)

An affordance point $\mathbf{p}^a$pa is encoded as $\langle \mathrm{apos}_{i^a} \rangle$⟨aposia​⟩, indicating a feasible local direction or landing location. An object point $\mathbf{p}^o$po is encoded as $\langle \mathrm{opos}_{i^o} \rangle$⟨oposio​⟩, localizing the task goal. The complete navigation output is serialized as:

$$\mathbf{y}_t = [ \langle \mathrm{apos}_{i_t^a} \rangle, \langle \mathrm{opos}_{i_t^o} \rangle, \langle \mathrm{act\_L0}_{k_{0,t}} \rangle, \langle \mathrm{act\_L1}_{k_{1,t}} \rangle, \langle \mathrm{act\_L2}_{k_{2,t}} \rangle ]$$yt​=[⟨aposita​​⟩,⟨oposito​​⟩,⟨act_L0k0,t​​⟩,⟨act_L1k1,t​​⟩,⟨act_L2k2,t​​⟩]

To avoid the mismatch between token prediction and geometric precision, each action chunk of 10 future SE(2) waypoints is tokenized using residual vector quantization.

The tokenizer contains 3 level-specific codebooks $\mathcal{C}^{(0)}, \mathcal{C}^{(1)}, \mathcal{C}^{(2)}$C(0),C(1),C(2), each with 256 codewords. The first level captures the coarse trajectory, and the next two levels successively quantize its residual:

$$k_\ell = \arg \min_k d_J \left(\mathbf{r}^{(\ell)}, \mathbf{e}_k^{(\ell)}\right)$$kℓ​=argkmin​dJ​(r(ℓ),ek(ℓ)​) $$\mathbf{r}^{(\ell+1)} = \mathbf{r}^{(\ell)} - \mathbf{e}_{k_\ell}^{(\ell)}, \quad \mathbf{r}^{(0)} = \mathbf{z}_t$$r(ℓ+1)=r(ℓ)−ekℓ​(ℓ)​,r(0)=zt​

where $d_J$dJ​ is a Jacobian-weighted trajectory distance. The trajectory distance is defined as:

$$d_{\mathrm{traj}}(\mathbf{z}, \hat{\mathbf{z}}) = \mathrm{ADE}(\mathbf{z}, \hat{\mathbf{z}}) + \lambda |\Delta \theta(\mathbf{z}) - \Delta \theta(\hat{\mathbf{z}})|$$dtraj​(z,z^)=ADE(z,z^)+λ∣Δθ(z)−Δθ(z^)∣

with $\lambda = 0.3$λ=0.3. For a generated prefix of length $L$L, the reconstruction is:

$$\hat{\mathbf{z}}_t^{(L)} = \sum_{\ell=0}^{L-1} \mathbf{e}_{k_\ell}^{(\ell)}, \quad L \in \{1, 2, 3\}$$z^t(L)​=ℓ=0∑L−1​ekℓ​(ℓ)​,L∈{1,2,3}

The training pipeline consists of three stages: Embodied-Reasoning Mid-training, Supervised Fine-tuning, and Online RL Post-training.

The authors begin with Embodied-Reasoning mid-training to specialize the backbone before introducing navigation actions. Stage I uses a task-balanced mixture spanning spatial reasoning, general VQA, image pointing, video QA, and more. This stage operates entirely through the native autoregressive interface. To prevent specialization from narrowing general competence, a specialize-then-retain curriculum is adopted. During the subsequent supervised stage, a rebalanced mixture is rehearsed together with navigation data.

Supervised Fine-tuning aligns the checkpoint with the unified navigation token space. The task-balanced optimization mixture combines retained reasoning and VQA data with instruction following, object navigation, and embodied visual tracking. Navigation examples are serialized using the same target structure: one $\langle \mathrm{apos}_i \rangle$⟨aposi​⟩ token and one $\langle \mathrm{opos} \rangle$⟨opos⟩ token followed by three RVQ tokens. Auxiliary reasoning examples and navigation trajectories are optimized with the shared causal language-model objective:

$$\mathcal{L}_{\mathrm{CE}} = - \sum_{j \in \mathcal{M}} \log p_\theta(y_j \mid \mathbf{x}, y_{<j})$$LCE​=−j∈M∑​logpθ​(yj​∣x,y<j​)

The navigation mixture additionally includes DAgger-collected examples to reduce the train-deployment state-distribution gap.

Finally, Online RL Post-training optimizes complete policy rollouts against task-level objectives using Group Relative Policy Optimization. For each episode seed, $G$G independent trajectories are rolled out and scored with a scalar reward $R(\tau^{(g)})$R(τ(g)). The within-group standardized reward serves as the advantage:

$$A^{(g)} = \frac{R(\tau^{(g)}) - \mu_R}{\sigma_R + \epsilon_{\mathrm{num}}}$$A(g)=σR​+ϵnum​R(τ(g))−μR​​

The surrogate objective is applied at the token level:

$$\mathcal{L}_{\mathrm{RL}} = - \mathbb{E} \left[ \min \left(\rho A, \mathrm{clip}(\rho, 1 - \varepsilon, 1 + \varepsilon) A \right) \right] + \beta D_{\mathrm{KL}}(\pi_\theta \| \pi_{\mathrm{ref}})$$LRL​=−E[min(ρA,clip(ρ,1−ε,1+ε)A)]+βDKL​(πθ​∥πref​)

Task-specific terminal rewards are defined for embodied visual tracking, instruction following, and object goal navigation, tailored to their respective success criteria.

Experiment

The model is evaluated on eight spatial-reasoning benchmarks, ten navigation settings spanning instruction following, object search, and visual tracking, and zero-shot deployment across game environments and four robot platforms. Using only a single forward RGB view, LightNav-0 matches or exceeds prior methods that rely on depth or panoramic inputs, while embodied-reasoning initialization and dual-channel pointing prove essential for robust goal-reaching. Scaling experiments show that broader environment coverage drives the largest improvements, with diminishing returns from additional data and model size beyond 4B parameters.

The Insight-Bench evaluation split comprises 1,097 episodes across 210 scenes, organized by five scene types and five spatial instruction mechanisms. Apartments dominate with 120 scenes and 239 episodes, while Extremum and Direction mechanisms are the most frequent instruction types. The distribution is deliberately uneven, with Ordinal and Relation episodes underrepresented, especially in House, Commercial, and Institution scenes. Apartments provide the largest share of scenes (120) and episodes (239), while Outdoor scenes are fewest (8) yet still contribute 228 episodes. Extremum instructions are the most numerous (250 episodes), whereas Ordinal instructions are the fewest (178), and Relation episodes drop sharply in House (31), Commercial (30), and Institution (32) scenes.

LightNav-ER, a 4B model, achieves the highest average score across eight embodied reasoning and spatial intelligence benchmarks, outperforming larger models such as the 8B Molmo2-ER. It leads on four tasks and is competitive on others, demonstrating that strong spatial understanding does not require larger scale. The 8B Molmo2-ER excels on point-based benchmarks but falls behind on certain spatial reasoning tasks, indicating that parameter count alone does not guarantee uniform gains. LightNav-ER (4B) attains a 67.4 average, surpassing the 8B Molmo2-ER (62.8) and other 4B models. It ranks first on RefSpatial, Where2Place, CV-Bench, and EmbSpatial, and second on Point-Bench and RoboSpatial VQA. Molmo2-ER (8B) leads on Point-Bench (77.3) but scores only 32.0 on RoboSpatial POI, the lowest among all compared methods. Qwen3-VL (4B) is the second-best overall, with a strong RoboSpatial POI score of 64.8.

Among multi-view methods evaluated on continuous vision-and-language navigation, Reborn achieves the highest success rates and path efficiency on both R2R and RxR unseen splits, while GridMM records the lowest navigation error and highest oracle success on R2R. Earlier approaches like CMA and Sim2Sim perform similarly to each other, and HPN+DN lags behind on the reported metrics. Reborn outperforms all other compared methods on R2R and RxR, with success rates and SPL substantially higher than CMA and Sim2Sim, and it more than doubles the RxR success rate. GridMM attains the lowest navigation error and highest oracle success on R2R, indicating strong goal localization even though its task success and path efficiency are slightly below Reborn.

The evaluation uses the Insight-Bench split with 1,097 episodes across five scene types and five instruction mechanisms, deliberately uneven in distribution. LightNav-ER, a 4B model, surpasses larger counterparts on embodied reasoning and spatial intelligence tasks, confirming that strong spatial understanding does not require larger scale. In continuous vision-and-language navigation, Reborn achieves the highest success and path efficiency, while GridMM demonstrates superior goal localization.