HyperAIHyperAI

Command Palette

Search for a command to run...

Jeveration: System-One-Modelle generieren jetzt

Yuhe Liu Qi Li Shihua Zhang Xinchao Wang

Zusammenfassung

System-One-Modelle wie Jev liefern Entscheidungen durch schnelle, parallele Vorhersage. In diesem Beitrag stellen wir Jeveration vor und zeigen, dass dasselbe Entscheidungsparadigma auch hochwertige Bilder in einem einzigen Vorwärtsdurchlauf erzeugen kann. Jeveration baut auf einem eingefrorenen Vision-Language-Modell auf und nutzt dessen visuell-semantisches Wissen für eine ganzheitliche Bildgenerierung. Die zentrale Herausforderung besteht darin, globale Konsistenz zu wahren. Ein einzelner Prompt lässt viele gültige Bilder zu, doch parallele Vorhersagen müssen zu einem kohärenten Ganzen konvergieren. Daher konditionieren wir alle Positionen auf einen gemeinsamen Zufallszustand, um die parallelen Vorhersagen zu koordinieren, sodass jede Ziehung ein anderes, aber kohärentes Bild ergibt. Da das Modell diesen Zustand unter direkter Bildsupervision tendenziell ignoriert, lernen wir zunächst eine Verteilung über diskrete Bildcodes, um die einstufige Generierung zu steuern. Über die Text-zu-Bild-Generierung hinaus vereinheitlichen wir zusätzlich Bild-zu-Bild-Generierung und dichte Vorhersage innerhalb derselben Architektur, sodass ein einziger Checkpoint alle drei Aufgaben unterstützt, jeweils in einem Vorwärtsdurchlauf. Experimente zeigen, dass Jeveration ein 512×512-Bild in 78 ms erzeugt – bis zu 122-mal schneller als bestehende vereinheitlichte Modelle – bei gleichzeitig wettbewerbsfähiger Bildqualität und Leistung in der dichten Vorhersage.

One-sentence Summary

Researchers at the National University of Singapore propose Jeveration, a System One model that repurposes a frozen vision-language model with a shared random state and a learned distribution over discrete image codes to generate coherent images in a single forward pass, unifying text-to-image generation, image-to-image generation, and dense prediction while producing 512×512512 \times 512512×512 images in 78 ms, up to 122×122 \times122× faster than existing unified models.

Key Contributions

  • Jeveration is introduced as a parallel decision model that generates 512×512 images in a single forward pass by repurposing a frozen vision-language model's visual-semantic knowledge.
  • To maintain global consistency, the method conditions all image code positions on a shared random state and learns a distribution over discrete image codes through token-level conditional matching with a masked model, avoiding the tendency to ignore the state under direct image supervision.
  • A single checkpoint unifies text-to-image generation, image-to-image generation, and dense prediction through shared adapters and task-specific output heads; experiments report 78 ms generation, up to 122× faster than existing unified models, with competitive image quality and dense prediction performance.

Introduction

Image generation is currently dominated by autoregressive and diffusion models, but both rely on many sequential model evaluations, which increases inference latency. Prior System One models can return typed visual decisions in parallel, but they have been applied mainly to understanding and scoring tasks rather than synthesis, and naive parallel image prediction tends to produce spatially incoherent regions. The authors introduce Jeveration, which adapts a frozen vision-language model with LoRA adapters and output heads to return an entire image as one joint decision in a single forward pass. To achieve coherence, all code positions are conditioned on a shared random state, and a masked model trained to learn the image distribution provides target conditionals for guiding the parallel decisions. The resulting single checkpoint covers text-to-image generation, image-to-image generation, and dense prediction tasks.

Method

Jeveration enables a vision-language model (VLM) to generate images through parallel typed decisions in a single forward pass, while simultaneously supporting dense prediction tasks within the same framework. For image synthesis, the model selects a codebook entry at each code position, whereas for dense prediction, it produces task-specific classes or scores. A shared random state provides common stochastic conditioning to all code positions, ensuring that each draw yields a different but globally coherent image.

Given a conditioning input yyy, such as a text prompt or a source image, Jeveration formulates the output as typed decisions. An image is represented as a joint choice of discrete codes c=(c1,…,cN)\boldsymbol{c} = (c_1, \dots, c_N)c=(c1​,…,cN​) from an image tokenizer. A direct approach that predicts a categorical distribution at each code position and samples codes independently in parallel fails to capture dependencies between codes. This independent factorization matches only the marginals, often resulting in codes that do not fit together to form a coherent image.

To overcome this, Jeveration draws one random state sequence ξ\xiξ per image and conditions every code position on this shared state:

qθ(c∣y)=Eξ∼p0[∏i=1Nqθ,i(ci∣y,ξ)]q_{\theta}(\boldsymbol{c} \mid y) = \mathbb{E}_{\xi \sim p_0} \left[ \prod_{i=1}^{N} q_{\theta, i}(c_i \mid y, \xi) \right]qθ​(c∣y)=Eξ∼p0​​[i=1∏N​qθ,i​(ci​∣y,ξ)]

Although the codes are sampled independently given (y,ξ)(y, \xi)(y,ξ), their shared dependence on ξ\xiξ allows the resulting image distribution to capture dependencies between positions. This enables the model to coordinate its code choices while deciding all positions in a single forward pass.

The authors implement these visual decisions by appending decision tokens to the input of a VLM. Text tokens encode the prompt and task instruction, and a frozen vision tower encodes any input image into visual tokens. The decision tokens are initialized from a sampled random state for image generation and serve as spatial queries for dense prediction. A frozen backbone with shared LoRA adapters processes the input and decision tokens in a single forward pass. In its full-attention layers, decision tokens attend to each other bidirectionally while the input prefix remains causal. Finally, task-specific heads map the decision tokens' final hidden states to image codes or dense predictions.

For image generation, the image tokenizer is a frozen TA-TiTok with N=128N = 128N=128 codes per image. The initial embeddings of the decision tokens encode the random state ξ∼p0\xi \sim p_0ξ∼p0​, which masks each position with probability ρ\rhoρ, fills the remaining positions with uniformly random codes, and adds Gaussian noise of scale σ\sigmaσ to all embeddings. A transformer head reads the final hidden states of these tokens together with ξ\xiξ and outputs a feature uiu_iui​ for each position. Each code is predicted from uiu_iui​ using two branches. The codebook branch fcbf_{\mathrm{cb}}fcb​ projects uiu_iui​ to the code space and applies a classifier initialized from the tokenizer's codebook. The wide branch fwidef_{\mathrm{wide}}fwide​, initialized at zero, maps uiu_iui​ directly to the code logits. The code distribution sums the two:

qθ,i(⋅∣y,ξ)=softmax(fcb(ui)+fwide(ui))q_{\theta, i}(\cdot \mid y, \xi) = \text{softmax} \left(f_{\mathrm{cb}}(u_i) + f_{\mathrm{wide}}(u_i)\right)qθ,i​(⋅∣y,ξ)=softmax(fcb​(ui​)+fwide​(ui​))

At inference, all codes are sampled in parallel as one joint decision and rendered by the tokenizer's decoder. For dense prediction, the decision tokens act as spatial queries aligned with the visual tokens of the input image, and task-specific heads decode each query's final hidden state into segmentation, depth, or surface-normal predictions.

The training procedure teaches the model to use ξ\xiξ to make coherent image decisions. Direct supervision on target codes does not encourage the model to use ξ\xiξ, since the random state is sampled independently of the target image. Therefore, the authors first learn a model of the image distribution through masked prediction, then use it to guide image decisions from random states.

In the first phase, the architecture is trained as a masked model with the VLM frozen. For target codes c\boldsymbol{c}c, a random set M\mathcal{M}M is masked, and the hidden codes are predicted from the visible codes cMˉ\boldsymbol{c}_{\bar{\mathcal{M}}}cMˉ​ and condition yyy:

Lmask(ϕ)=E[1∣M∣∑i∈M−log⁡pϕ,i(ci∣cMˉ,y)]\mathcal{L}_{\mathrm{mask}}(\phi) = \mathbb{E} \left[ \frac{1}{|\mathcal{M}|} \sum_{i \in \mathcal{M}} -\log p_{\phi, i}(c_i \mid \boldsymbol{c}_{\bar{\mathcal{M}}}, y) \right]Lmask​(ϕ)=E[∣M∣1​i∈M∑​−logpϕ,i​(ci​∣cMˉ​,y)]

The visible codes supply image-specific context, enabling the model to learn dependencies between positions.

In the second phase, the decision model θ\thetaθ starts from the first-phase model and learns to turn random states into whole images. From (y,ξ)(y, \xi)(y,ξ), θ\thetaθ predicts logits and samples a code sequence c^\hat{\boldsymbol{c}}c^ in a single pass. A random subset M\mathcal{M}M of c^\hat{\boldsymbol{c}}c^ is re-masked, and two masked models predict the hidden codes from the same visible codes. The frozen first-phase model ϕ\phiϕ represents the image distribution, while an auxiliary model ψ\psiψ tracks the current distribution of θ\thetaθ. The auxiliary model begins as a copy of ϕ\phiϕ and continues training on θ\thetaθ's samples. The two predictions agree when θ\thetaθ matches the image distribution. The authors use their difference to update θ\thetaθ by minimizing:

LDM(θ)=E[1∣M∣∑i∈Msg(pψ,i−pϕ,i)⊤ℓθ,i]\mathcal{L}_{\mathrm{DM}}(\theta) = \mathbb{E} \left[ \frac{1}{|\mathcal{M}|} \sum_{i \in \mathcal{M}} \mathrm{sg}(p_{\psi, i} - p_{\phi, i})^\top \ell_{\theta, i} \right]LDM​(θ)=E[∣M∣1​i∈M∑​sg(pψ,i​−pϕ,i​)⊤ℓθ,i​]

where sg\mathrm{sg}sg stops gradients. This pushes θ\thetaθ to coordinate its codes through the shared state ξ\xiξ.

In the final part of the second phase, confidence-guided correction adds direct supervision on generated samples. For some samples in each batch, the half of the positions with the lowest confidence are re-masked, and the frozen model completes them over several rounds into cref\boldsymbol{c}^{\mathrm{ref}}cref. The model θ\thetaθ is then trained to predict these corrected codes from the same (y,ξ)(y, \xi)(y,ξ):

Lgen=LDM+λLcorr,Lcorr=Eselected[1∣R∣∑i∈R−log⁡qθ,i(ciref∣y,ξ)].\begin{array}{l} \mathcal{L}_{\mathrm{gen}} = \mathcal{L}_{\mathrm{DM}} + \lambda \mathcal{L}_{\mathrm{corr}}, \\ \mathcal{L}_{\mathrm{corr}} = \mathbb{E}_{\mathrm{selected}} \left[ \frac{1}{|\mathcal{R}|} \sum_{i \in \mathcal{R}} -\log q_{\theta, i}(c_i^{\mathrm{ref}} \mid y, \xi) \right]. \end{array}Lgen​=LDM​+λLcorr​,Lcorr​=Eselected​[∣R∣1​∑i∈R​−logqθ,i​(ciref​∣y,ξ)].​

Dense tasks are supervised with ground truth, using cross-entropy for segmentation, an ℓ1\ell_1ℓ1​ depth error, and 1−cos⁡1 - \cos1−cos for surface normals. The frozen model, the auxiliary model, and the correction are used only in training, ensuring that inference remains a single forward pass of θ\thetaθ.

Experiment

The experiments evaluate a single Jeveration checkpoint trained with rank-256 LoRA adapters on a frozen VLM for text-to-image generation, image-to-image tasks, and dense prediction, using generation benchmarks such as GenEval, DPG-Bench, and HPSv2.1, along with structure fidelity, NYUv2, and latency metrics. Text-to-image results show that the model generates coherent images in one forward pass and is faster than both unified and generation-only baselines while maintaining competitive quality. Image-to-image and dense prediction experiments demonstrate that the same checkpoint handles controllable generation, segmentation, and depth estimation with strong structural fidelity and little added latency. Ablations confirm that the shared random state, correction target, dual image-head branches, source conditioning, and joint training each contribute to image quality, controllability, or task flexibility.

Jeveration generates 512x512 images in a single forward pass with the lowest end-to-end latency in the comparison and stronger GenEval than the listed unified and same-resolution generation-only baselines. It is roughly two orders of magnitude faster than autoregressive Janus-Pro baselines and outperforms several one-step generation-only models on GenEval despite their higher output resolution. The results indicate a frozen vision-language backbone can provide competitive text-to-image quality without iterative sampling. At 512-px resolution, it achieves higher GenEval than multi-step generation-only models while being several times faster end to end. It is roughly two orders of magnitude faster than the autoregressive Janus-Pro baselines at batch 1 on the same hardware.

On held-out image-to-image examples, Jeveration achieves strong structure fidelity across edge, depth, and colorization tasks. It improves over a decoder without source conditioning, particularly in colorization and edge fidelity, and achieves higher style PSNR. Jeveration attains the best edge and depth fidelity among the compared conditions. Source-image conditioning sharply improves colorization fidelity over source-free decoding and lifts edge fidelity above direct input copying. Jeveration achieves better style PSNR than the decoder without source, indicating closer alignment with paired style targets.

On NYUv2 dense prediction, the unified model matches dense-only segmentation accuracy from the same backbone and far exceeds DeepLabV3+. Depth accuracy is close to the dense-only baselines, while surface normal error is somewhat higher. Each prediction is produced in a single forward pass, supporting one checkpoint for both image generation and dense prediction. The unified model matches dense-only segmentation scores from the same backbone while outperforming DeepLabV3+ by a wide margin. Depth error is close to the dense-only baselines and much lower than DeepLabV3+, but surface normal accuracy trails the dense-only models.

Ablations indicate that a shared random state improves both composition and image quality over a fixed state. A correction target adds a further small quality gain while leaving composition similar, and combining wide and codebook image-head branches outperforms either branch alone. The results support the selected configuration for coherent, high-quality generation. A shared random state yields higher GenEval and HPS than a fixed state, indicating better coherence and image quality. Using a correction target improves HPS while GenEval remains similar, suggesting it refines details rather than composition. Removing the correction target slightly lowers image quality with no meaningful change in composition. The wide plus codebook image-head combination outperforms the wide-only and codebook-only variants.

Joint training across text-to-image, image-to-image, and dense prediction tasks preserves text-to-image quality almost entirely. The all-task model performs on par with a text-to-image only baseline on GenEval, HPS, and DPG metrics. The added tasks introduce little interference while enabling dense prediction and image-to-image generation. All-task training matches text-to-image-only quality, with only marginal differences across GenEval, HPS, and DPG. Adding image-to-image and dense tasks comes at almost no cost to text-to-image performance. For dense prediction, joint training matches dense-only segmentation and shows only a modest decline in depth and normal accuracy.

Jeveration is evaluated as a unified vision-language model across text-to-image, image-to-image, and dense prediction tasks. It produces 512x512 images in a single forward pass with lower latency and stronger GenEval than several generation-only baselines, while image-to-image experiments show high edge, depth, and colorization fidelity that improves with source conditioning. For dense prediction, the unified model matches segmentation quality from the same backbone and outperforms DeepLabV3+, with depth close to dense-only baselines; ablations confirm that shared random states, the correction target, and combined image-head branches improve quality, and joint training preserves text-to-image performance with little interference.


KI mit KI entwickeln

Von der Idee bis zum Launch – beschleunigen Sie Ihre KI-Entwicklung mit kostenlosem KI-Co-Coding, sofort einsatzbereiter Umgebung und bestem GPU-Preis.

KI-gestütztes kollaboratives Programmieren
Sofort einsatzbereite GPUs
Die besten Preise

HyperAI Newsletters

Abonnieren Sie unsere neuesten Updates
Wir werden die neuesten Updates der Woche in Ihren Posteingang liefern um neun Uhr jeden Montagmorgen
Unterstützt von MailChimp