One-sentence Summary

Researchers from CMLab, Chung-Ang University and Kyung Hee University introduce Self-Geometry, a GT-free, plug-and-play test-time adaptation pipeline that imposes explicit multi-view geometric constraints from $2D$2D pixel correspondences by combining Geometric Disentanglement Optimization, which uses Multi-View Consistency and Epipolar Consistency losses with gradient disentanglement to prevent gradient conflict, a Frame Angular-Neighbor view sampler based on $SO(3)$SO(3) geodesic distances, and lightweight LoRA adaptation, achieving consistent improvements in pose and geometry estimation across six vision foundation models (VGGT, π3, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).

Key Contributions

• Self-Geometry is a plug-and-play test-time adaptation pipeline that imposes explicit multi-view geometric constraints on pretrained vision foundation models using 2D pixel correspondences as pseudo ground truth, without GT annotations or teacher distillation.
• The pipeline consists of Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with gradient disentanglement to avoid gradient conflict; Frame Angular-Neighbor, an SO(3) geodesic distance based view sampler; and Lightweight TTA, which adapts models via LoRA.
• Experiments across six pretrained vision foundation models (VGGT, π3, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom) show consistent improvements in both pose and geometry estimation, with per-scene adaptation completed within two minutes on a single NVIDIA RTX PRO 6000 GPU.

Introduction

Multi-view 3D reconstruction aims to recover camera poses and dense scene geometry from multi-view images, and recent Vision Foundation Models (VFMs) such as VGGT, π³, and Depth Anything 3 provide feed-forward zero-shot prediction of depths, poses, and pointmaps without per-scene optimization. However, these models are pretrained without explicit multi-view geometric constraints because enforcing bundle adjustment is costly, so their predictions can violate geometric consistency; prior test-time adaptation methods often rely on implicit self-consistency, fine-tuning data, teacher distillation, or specific architectures, which limits their accuracy and generality. The authors propose Self-Geometry, a ground-truth-free and plug-and-play test-time adaptation pipeline that uses 2D pixel correspondences from an external matcher as pseudo ground truth to impose explicit multi-view geometric supervision. It combines point-to-point and point-to-line losses with gradient disentanglement, an SO(3)-guided view sampler, and lightweight LoRA adaptation, improving pose and geometry estimation across six pretrained VFMs and four datasets in about two minutes per scene.

Method

Pretrained Vision Foundation Models (VFMs) are trained without enforcing explicit multiview geometric consistency, which can lead to inconsistency in the resulting pointmaps. As shown in the figure below, implicit self-consistency methods fail to substantially improve camera pose and depth, whereas the proposed Self-Geometry directly imposes explicit multiview geometric constraints to yield consistent improvements.

The key insight driving this approach is that 2D pixel correspondences themselves define explicit multiview geometric supervision on the predictions of a pretrained VFM. To resolve the inconsistency, the authors present Self-Geometry, a GT-free and Plug-and-play Test-Time Adaptation (TTA) pipeline.

As depicted in the figure below, the proposed Self-Geometry consists of three complementary components: Geometric Disentanglement Optimization (GDO), Frame Angular-Neighbor (FAN), and Lightweight Test-Time Adaptation (Lightweight TTA).

The GDO imposes explicit multiview geometric constraints through a structured optimization process. First, Scene Initialization extracts pseudo-correspondences from input views using an external feature matcher. Given a set of input views $\mathcal{T} = \{\mathbf{I}_i\}_{i=1}^N$T={Ii​}i=1N​, the pretrained VFM predicts camera poses $\mathcal{P} = \{\mathbf{P}_i\}_{i=1}^N$P={Pi​}i=1N​, depth maps $\mathcal{D} = \{\mathbf{D}_i\}_{i=1}^N$D={Di​}i=1N​, and pointmaps $\mathcal{X} = \{\mathbf{X}_i\}_{i=1}^N$X={Xi​}i=1N​. The pseudo-correspondence set is defined as:

Second, Pseudo-Correspondence Filtering removes mismatched pseudo-correspondences by sequentially applying an Epipolar Consistency (EC) Loss-based filter and a Multi-View Consistency (MVC) Loss-based filter to ensure reliable supervision.

Third, the Multi-View Consistency Loss ($\mathcal{L}_{\mathrm{mvc}}$Lmvc​) formalizes the point-to-point reprojection constraint:

This loss jointly supervises camera poses and depths but inherits pose-depth ambiguity.

Fourth, to resolve this ambiguity, the Epipolar Consistency Loss ($\mathcal{L}_{\mathrm{ec}}$Lec​) introduces a depth-independent point-to-line constraint:

This loss supervises camera poses in a depth-independent manner.

Fifth, Gradient Disentanglement (GD) prevents gradient conflict between the two losses on camera poses by projecting $\nabla \mathcal{L}_{\mathrm{mvc}}$∇Lmvc​ onto the orthogonal complement of $\nabla \mathcal{L}_{\mathrm{ec}}$∇Lec​:

This projection ensures the two losses operate complementarily at the gradient level.

To efficiently apply these constraints, FAN samples input views using scene-scale-invariant SO(3) geodesic distances. The SO(3) geodesic distance between views $i$i and $j$j is calculated as:

Geometry-Rich View Selection (GRV) selects the target view by maximizing the SO(3) bin entropy:

Angular-Neighbor Sampling (ANS) then samples source views from each SO(3) bin to ensure uniform scene coverage without the need for heuristic per-scene normalization.

The Lightweight TTA specializes the pretrained VFM to the target scene without additional training data, preserving the geometric prior acquired during pretraining. It employs a LoRA-based adapter inserted into the QKV weights of the attention blocks, keeping the pretrained VFM frozen and updating only the LoRA parameters. This parameter-efficient strategy enables per-scene adaptation on a single GPU within two minutes.

Experiment

This work evaluates Self-Geometry for pose and geometry estimation across multiple pretrained vision foundation models, including VGGT, π3, and DA3, on 7Scenes, ETH3D, ScanNet++, and HiRoom, comparing against Free-Geometry and TCO under the DA3 protocol. The method consistently improves average pose and geometry results, remains robust where TCO collapses on π3, and shows qualitative reductions in depth errors and cleaner surface reconstructions. Ablations confirm that combining epipolar and multi-view consistency losses with geometric disentanglement is essential, pseudo-correspondence filtering improves precision, and the frame angular-neighbor sampler is robust across hyperparameters. Complexity analysis shows that LoRA adaptation adds only 0.7 to 3.4 percent extra parameters and completes per-scene adaptation within practical limits.

The comparison evaluates multi-view 3D visual foundation models on ground-truth-free adaptation, teacher-free operation, plug-and-play use, and explicit geometry. Frozen and fine-tuning-based approaches are generally not ground-truth-free and provide limited explicit geometry or plug-and-play support. Test-time adaptation methods split between TCO, which is teacher-free, plug-and-play, and explicit-geometry but not ground-truth-free, and approaches like TTT3R and Online3R, which are ground-truth-free and teacher-free but not plug-and-play. Frozen and fine-tuning-based models such as VGGT, π3, and DA3 are not ground-truth-free and lack explicit geometry support. Among test-time adaptation methods, TCO uniquely combines teacher-free, plug-and-play, and explicit-geometry support but remains not ground-truth-free, while TTT3R and Online3R are ground-truth-free and teacher-free but not plug-and-play.

On ETH3D with VGGT, the GDO ablations show that the choice of loss components and gradient disentanglement strategy leads to mixed effects across pose and geometry metrics. The variant without the epipolar consistency loss gives the strongest geometry performance and high pose accuracy, while the variant without the multi-view consistency loss weakens several metrics. Gradient disentanglement improves the strictest pose metric but reduces looser pose accuracy and unpaced geometry relative to no disentanglement, indicating a precision-focused trade-off. The variant without the epipolar consistency loss achieves the best F1 with pacing and ties for the best AUC@30 and F1 without pacing among the compared settings. Removing the multi-view consistency loss lowers pose and geometry metrics compared with removing the epipolar consistency loss. Gradient disentanglement improves AUC@1 but decreases AUC@3, AUC@30, and F1 without pacing relative to the no-disentanglement setting. The baseline remains competitive on some metrics and is not uniformly outperformed by every ablated variant.

Filtering pseudo-correspondences improves precision and downstream pose and geometry, with the combined L_ec plus L_mvc filter delivering the best overall downstream performance. The raw unfiltered variant retains all matches but has much lower precision and downstream scores, while L_mvc alone has the strongest F1 among the filtered variants. The results indicate that precision gains from complementary filtering stages matter more for adaptation than preserving recall. The combined L_ec plus L_mvc filtering variant achieves the highest precision and the best downstream pose and geometry metrics among the filtering ablations. Raw pseudo-correspondences have perfect recall but much lower precision and substantially worse downstream performance, while L_mvc alone provides the strongest F1 among the variants.

FAN variants improve over the baseline on ETH3D across pose AUC and geometry F1. The SO(3) bin width and source view ordering have only a modest effect on performance, indicating robustness in these sampling choices. Target view selection is the dominant factor, with a static globally selected target view giving the strongest reported gains while dynamic updates underperform. SO(3) bin width and source view ordering have limited impact: 15, 30, and 45 degree settings and different orderings all perform comparably. Target view selection is the dominant factor: GRV improves substantially over a fixed target view, while dynamic GRV underperforms GRV.

Across the six pretrained VFMs, Self-Geometry adds only a small LoRA parameter overhead, from under one percent to a few percent. Initialization and FAN stages contribute little to per-scene adaptation time, while the GDO stage dominates and scales with model size. Total per-scene time remains practical across ETH3D and the other datasets, with smaller DA3 models adapting faster than larger models. LoRA parameter overhead is modest across all models, staying below four percent of the pretrained VFM parameters. Initialization time is roughly constant around 0.3 minutes regardless of model size, and FAN time is negligible. GDO accounts for most adaptation time and grows with model size, while smaller DA3 variants adapt much faster than larger models. Total per-scene adaptation remains practical across ETH3D, 7Scenes, ScanNet++, and HiRoom, with the smallest model being the fastest.

The experiments evaluate multi-view 3D foundation models on ground-truth-free adaptation, teacher-free and plug-and-play operation, and explicit geometry, positioning Self-Geometry relative to frozen, fine-tuning, and test-time adaptation approaches. Ablations on ETH3D show that pseudo-correspondence filtering is critical for downstream pose and geometry, with the combined epipolar and multi-view filtering giving the best precision, while loss component choices and gradient disentanglement produce mixed precision-oriented trade-offs. FAN studies indicate target view selection is the dominant factor, whereas SO(3) bin width and source view ordering have only modest effects. Across six pretrained models, Self-Geometry adds a small LoRA overhead and practical per-scene adaptation time, with the gradient-based stage dominating the cost.