One-sentence Summary

Researchers from Brown University, New York University, and AMI Labs prove a Platonic Representation Theorem showing that Joint-Embedding Predictive Architectures trained separately on any number of modalities with good views learn the same representation up to one rotation, and they demonstrate that next-token prediction is an inefficient JEPA with bad views by showing a 109M-parameter text JEPA outperforms language models up to 72B parameters on both alignment and zero-shot benchmarks.

Key Contributions

• A Platonic Representation Theorem proves that Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation when positive pairs are two observations of one scene and misalignment is otherwise bounded.
• Next-token prediction is identified as a JEPA with bad views, where positives are parts of a single document; this explains why language models align with vision only through scale, and experiments show good views require 42 times fewer training samples than next-token prediction on the same corpus.
• Empirical validation spans 145k captions to 940M web images and uncovers a scaling law of alignment that predicts unseen runs within seed noise. A 109M-parameter text JEPA aligns with DINOv2 and CLIP image encoders better than language models up to 72B parameters and CLIP's text encoder, and one learned rotation turns separately trained encoders into an image captioner with CIDEr .838 on COCO; on the pair-free benchmark, this text JEPA also beats every reported zero-shot accuracy with language models up to 8B parameters, e.g., 64% against at most 56% top-5 on ImageNet-100.

Introduction

Deep networks have revived an old question: whether separate observers of the same world converge to the same representation. Empirically, word embeddings and vision-language models show increasing cross-modal alignment, a phenomenon Huh et al. called the Platonic Representation Hypothesis, but that remained an observation with no proof for practical self-supervised objectives such as Joint-Embedding Predictive Architectures, or JEPAs. The authors turn this observation into a theorem for JEPAs. They show that when separately trained JEPAs use positive pairs that relate the same world states, their learned representations coincide up to a rotation; when positives corrupt a single observation instead, encoders spend capacity on modality-specific nuisances that cannot be rotationally aligned. This reframes the hypothesis as a statement about positive views and provides a training principle, with empirical results showing good-view JEPAs align better and more data-efficiently than next-token prediction.

Method

The authors formalize a world observed by multiple modalities through injective renderings. A Joint-Embedding Predictive Architecture (JEPA) for a specific modality is trained on positive pairs drawn from a distribution. The core objective minimizes a prediction term plus an anti-collapse term. The authors prove the Platonic Representation Theorem, which states that if multiple modalities share the same positive-pair distribution over world states, their encoders learn the same embedding of the world up to a single rotation. This holds for spectral JEPAs with second-order anti-collapse, which learn the top eigenvectors of the graph defined by the positive pairs.

The quality of alignment depends critically on the nature of the positive pairs. The authors distinguish between good-view and bad-view JEPAs. Good-view JEPAs use positive pairs consisting of two independent observations of the same world state, such as two photographs of a scene or two captions describing it. These JEPAs ignore modality-specific nuisance and learn a function of the world state alone. Bad-view JEPAs use positive pairs that are corruptions of a single observation, such as two crops of an image or two maskings of a sentence. These JEPAs tend to spend embedding dimensions on the nuisance shared by the corruptions, which cannot be aligned across modalities via rotation.

As shown in the figure below, good views induce the world's own kernel by design, leading to perfect alignment up to rotation, whereas bad views and next-token prediction rely on data density to connect states and often mix in nuisance. The right panel demonstrates that a good-view JEPA achieves the same alignment with DINOv2 as next-token prediction but requires 42 times fewer samples.

Furthermore, the authors illustrate how different positive pair definitions connect observations. Good views connect every description of a scene without crossing into others. Bad views and next-token prediction create sparse links within scenes or spill across scenes due to shared nuisance.

In practice, modalities rarely share the exact same positive pairs. The authors provide a bound on misalignment based on the mismatch between the kernels induced by the positive pairs of different modalities. The root-mean-square distance between embeddings after the best rotation is bounded by the difference in their kernels and the spectral gap.

As shown in the figure below, this bound is validated empirically on randomly generated finite worlds, showing that the error consistently lies below the theoretical bound.

The authors also analyze the trade-off between the reach of bad views and the amount of data required. Weaker positives require significantly more samples per world state to achieve alignment, as shown in the figure below.

The authors formalize next-token prediction as a JEPA. The positives are a prefix and its continuation within a single document. Because a prefix and its continuation share the document's specific nuisance, next-token prediction acts as a bad-view JEPA. It spends embedding directions on this nuisance, explaining why language models require massive scale to align with vision models compared to good-view JEPAs.

Leveraging the theorem that representations agree up to a rotation, the authors design a captioning system that requires no network training on image-caption pairs. A text JEPA and a causal transformer decoder are trained on text alone. A frozen vision encoder is used for images. A single rotation matrix is fitted on a small set of image-caption pairs to align the frozen image embeddings with the text embedding space. To caption an image, it is embedded, rotated, and decoded. Gaussian noise is added to the embedding to tolerate residual alignment error.

The figure below demonstrates captions generated by this system on COCO test images. The system uses frozen encoders joined by one rotation, yet it accurately describes scenes, objects, and actions.

Experiment

These experiments evaluate separately trained image and text encoders with a JEPA objective, measuring cross-modal alignment on Flickr30k, MSR-VTT, COCO, and large web-text corpora through retrieval, mutual nearest neighbors, and rotation-based mapping. They validate that good views drawn from different observations of the same world state and then trimmed consistently align better than bad views or next-token prediction on identical data, with trimming being the main factor and benefits appearing on both modalities. The results further show that good-view JEPAs scale far more sample-efficiently than next-token prediction, outperform much larger language models, approach pair-trained encoders, and enable captioning and alignment with little or no paired image-caption supervision.

A text encoder trained with good-view JEPA aligns with an image JEPA more strongly than encoders trained with bad-view JEPA or next-token prediction on Flickr test pairs. The advantage appears across mutual nearest-neighbor overlap and rotation-fitted retrieval metrics. Training next-token prediction on pairs of captions from the same image improves alignment only slightly, indicating the JEPA objective drives the gain. Caption views beat masked spans and next-token prediction on mutual 10-NN, R@1, and R@5, with gaps larger than seed variation. Next-token prediction trained on pairs of captions from the same image barely improves over standard next-token prediction, so the alignment advantage comes from the JEPA objective.

When paired with a frozen DINOv2 ViT-B/14, the good-view JEPA text encoder shows the strongest alignment among the listed models, with retrieval scores roughly double those of the bad-view JEPA and next-token baselines. It also surpasses the pretrained Qwen2.5-0.5B and BERT-base text encoders on the same metrics. Fitting the rotation on 30k Flickr train and validation pairs further improves retrieval, and the good-view JEPA remains the best of the authors' models. Good-view JEPA achieves the highest mutual 10-NN and R@1 among all listed text encoders, with R@1 on 1k pairs about twice that of the bad-view JEPA and next-token models. Good-view JEPA also exceeds the pretrained Qwen2.5-0.5B and BERT-base retrieval scores against frozen DINOv2, and its advantage widens when the rotation is fit on 30k pairs.

A 109M-parameter good-view JEPA trained on 900M image-derived text samples aligns more strongly with DINOv2 and CLIP vision encoders than language models up to 72B parameters. The advantage appears across mutual 10-NN, linear CKA, and COCO retrieval, while next-token prediction and the bad-view JEPA trained on the same texts fall far behind. The good-view objective, rather than the text data alone, appears to drive the alignment gain. The good-view JEPA surpasses all listed language models up to 72B parameters on mutual 10-NN agreement with DINOv2 ViT-B and CLIP ViT-H/14. Next-token prediction and bad-view JEPA on identical texts trail far behind, reinforcing the role of the view design.

A 109M good-view JEPA aligns vision and text without paired examples better than prior pair-free methods, achieving the lowest FOSCTTM and the highest zero-shot classification accuracies in the comparison. Under identical conditions it matches or outperforms a rerun of Qwen3-8B, while next-token prediction on the same texts performs near chance on FOSCTTM and much worse on zero-shot accuracy. This indicates that good-view JEPA representations, not next-token text training, drive geometry-based alignment. The 109M good-view JEPA surpasses every zero-shot accuracy previously reported for pair-free alignment and matches the best prior FOSCTTM on these data. It matches the rerun Qwen3-8B on FOSCTTM and CIFAR-10 and beats it by clear margins on CIFAR-100 and ImageNet-100. The bad-view JEPA also aligns far better than next-token prediction, while next-token prediction on the same texts fails on FOSCTTM and CIFAR-10. A smaller 33M good-view JEPA still exceeds the best prior pair-free zero-shot accuracies.

Across Flickr retrieval, frozen DINOv2 alignment, scaling comparisons, and pair-free vision-text alignment, the authors compare good-view JEPA text encoders with bad-view JEPA, next-token prediction, pretrained language models, and prior pair-free methods. The good-view JEPA objective consistently yields stronger cross-modal alignment and retrieval, often exceeding much larger language models and prior pair-free results, while next-token and bad-view baselines remain weak. These gains come from the JEPA view design rather than the text data or paired captions.