HyperAIHyperAI

Command Palette

Search for a command to run...

تقطير المطابقة البعدية

Hidir Yesiltepe Julius Berner Arash Vahdat Pinar Yanardag

الملخص

يعمل تقطير الانتشار على تسريع توليد الصور والفيديو من خلال ضغط عملية أخذ العينات التكرارية لنموذج المعلم المُدرَّب مسبقًا إلى عدد قليل من تقييمات الشبكة. تحقق الطرق القائمة على التوزيع ذلك عن طريق مطابقة توزيع مخرجات الطالب مع توزيع المعلم. وغالبًا ما تُدار هذه المطابقة بأهداف تباعد كولباك-لايبلر العكسي، التي يؤدي انحيازها نحو البحث عن الأنماط إلى تفضيل أنماط المعلم المهيمنة على حساب التنوع عبر العينات. نقدم تقطير المطابقة البعدية (PMD)، وهو عضو جديد في عائلة التقطير القائمة على التوزيع يجمع بين أخذ العينات البعدية المغطية للأنماط والتوليد بأي عدد من الخطوات. عند انعدام خسارة المجتمع، يستعيد PMD التوزيع البعدي الكامل للمعلم للصور النظيفة عند كل مستوى ضوضاء مُدرَّب عليه. ويدعم طالبه العشوائي أي ميزانية خطوات أخذ عينات ضمن شبكة الخطوات الزمنية المُدرَّب عليها دون تدريب إضافي. لتعلم هذه التوزيعات البعدية، يطابق PMD متوسطات مُعاد ترجيحها أسيًا لعينات الطالب المشروطة بحالة ضوضاء ثابتة مع متوسطات التوزيع البعدي المقابلة للمعلم. وفي ظل إفساد غاوسي، يكون متوسط التوزيع البعدي المُمال أسيًا عبارة عن تقييم واحد للمعلم عند مدخل مُزاح. نُثبت أن مطابقة المتوسطات المُمالة على جوار مفتوح تحدد التوزيع البعدي الكامل بشكل فريد. وهكذا يشرف PMD على التوزيع الشرطي بأكمله من خلال عائلة من قيود المتوسطات، دون أي شبكة مساعدة لدرجات الطالب. تُظهر التجارب على SDXL وFLUX.2 [klein] 4B أن PMD يحقق أفضل FID بين الطرق ذات الأربع خطوات على كلا النموذجين، مع الحفاظ على أداء تنافسي أو متصدر عبر مقاييس التوافق مع الموجهات وجودة الإدراك.

One-sentence Summary

Researchers from Virginia Tech and NVIDIA introduce Posterior Matching Distillation (PMD), a distribution-based diffusion distillation method that matches exponentially reweighted student-sample averages to teacher posterior means and thereby uniquely identifies the full posterior without an auxiliary student-score network, achieving the best 4-step FID on both SDXL and FLUX.2 [klein] 4B.

Key Contributions

  • The paper introduces Posterior Matching Distillation (PMD), a distribution-based distillation method that combines mode-covering posterior sampling with any-step generation. At zero population loss, PMD recovers the teacher's full posterior of clean images at each trained noise level, and its stochastic student supports any-step sampling within the trained timestep grid without further training.
  • PMD supervises the full conditional posterior by matching exponentially reweighted averages of student samples conditioned on a fixed noisy state to corresponding teacher posterior means. Under Gaussian corruption, each exponentially tilted posterior mean is obtained from a single frozen-teacher evaluation at a shifted input, and matching these tilted means over an open neighborhood is proven to uniquely identify the full posterior without an auxiliary student-score network.
  • Experiments on SDXL and FLUX.2 [klein] 4B show that PMD attains the best FID among 4-step methods on both models while remaining competitive or leading on prompt-alignment and perceptual-quality metrics.

Introduction

Diffusion and flow models are powerful generative models for images, video, and text, but their iterative denoising process typically requires many neural network evaluations. Distillation reduces this cost by training a one- or few-step student, yet prior work differs in what it preserves: trajectory-based methods compress the teacher’s sampling path, while distribution-based methods such as Distribution Matching Distillation and Moment Matching align generated marginals or conditional means. These approaches can still miss conditional structure, for example DMD’s approximate reverse-KL objective can be mode-seeking and reduce diversity, and Moment Matching compares means at a re-noised student prediction rather than at the original noisy input. The authors propose Posterior Matching Distillation (PMD), which directly matches the teacher’s conditional posterior at each noise level using exponentially tilted means. A key Tilt=Shift identity allows each target to be obtained from a single frozen-teacher evaluation at a shifted noisy input, avoiding expensive teacher posterior sampling while providing constraints that uniquely identify the full multimodal conditional posterior.

Method

The authors introduce Posterior Matching Distillation (PMD) to distill a diffusion teacher into a few-step student by matching the full conditional denoising posterior rather than just the posterior mean. Under the Gaussian corruption process, the conditional distribution over clean images for a noisy observation ztz_tzt​ is given by q(x∣zt)∝q0(x)exp⁡(−∥zt−αtx∥2/2σt2)q(x \mid z_t) \propto q_0(x) \exp(-\|z_t - \alpha_t x\|^2 / 2\sigma_t^2)q(x∣zt​)∝q0​(x)exp(−∥zt​−αt​x∥2/2σt2​). At high noise levels, this posterior can be broad or multimodal, making it crucial to model the conditional uncertainty for few-step generation.

To capture information beyond the posterior mean, the authors consider an exponential tilt of the teacher's conditional distribution using a tilt vector λ∈Rd\lambda \in \mathbb{R}^dλ∈Rd. The tilted distribution is defined as qλ(x∣zt)=eλ⊤xq(x∣zt)Eq(x∣zt)[eλ⊤x]q_\lambda(x \mid z_t) = \frac{e^{\lambda^\top x} q(x \mid z_t)}{\mathbb{E}_{q(x \mid z_t)}[e^{\lambda^\top x}]}qλ​(x∣zt​)=Eq(x∣zt​)​[eλ⊤x]eλ⊤xq(x∣zt​)​. As shown in the figure below:

The exponential reweighting factor eλ⊤xe^{\lambda^\top x}eλ⊤x changes the relative prominence of the posterior's modes, giving greater weight to samples aligned with λ\lambdaλ. Different tilts emphasize different modes of the same posterior. Computing the means of these tilted distributions would typically require sampling and reweighting. However, the authors establish the Tilt=Shift Identity, which states that for a fixed noise level, exponentially tilting the posterior is exactly equivalent to shifting the noisy input to the denoiser: qλ(x∣zt)=q(x∣zt+ρtλ)q_\lambda(x \mid z_t) = q(x \mid z_t + \rho_t \lambda)qλ​(x∣zt​)=q(x∣zt​+ρt​λ), where ρt=σt2/αt\rho_t = \sigma_t^2 / \alpha_tρt​=σt2​/αt​. Consequently, the mean of the tilted posterior is obtained directly from the frozen teacher denoiser as Eqλ(x∣zt)[x]=Dt(zt+ρtλ)\mathbb{E}_{q_\lambda(x \mid z_t)}[x] = D_t(z_t + \rho_t \lambda)Eqλ​(x∣zt​)​[x]=Dt​(zt​+ρt​λ).

The authors prove that matching these tilted means over an open neighborhood of λ=0\lambda = 0λ=0 is sufficient to identify the entire conditional distribution. Let GθG_\thetaGθ​ denote a stochastic student that predicts a clean image x^=Gθ(zt,u,t)\hat{x} = G_\theta(z_t, u, t)x^=Gθ​(zt​,u,t) with random input u∼N(0,I)u \sim \mathcal{N}(0, I)u∼N(0,I). PMD requires the student's exponentially weighted mean to match the teacher's tilted mean:

Eu[x^eλ⊤x^]Eu[eλ⊤x^]=Dt(zt+ρtλ)\frac{\mathbb{E}_u[\hat{x} e^{\lambda^\top \hat{x}}]}{\mathbb{E}_u[e^{\lambda^\top \hat{x}}]} = D_t(z_t + \rho_t \lambda)Eu​[eλ⊤x^]Eu​[x^eλ⊤x^]​=Dt​(zt​+ρt​λ)

If this holds for every tilt in the neighborhood, the student recovers the teacher's full conditional posterior pGθ(⋅∣zt)=q(⋅∣zt)p_{G_\theta}(\cdot \mid z_t) = q(\cdot \mid z_t)pGθ​​(⋅∣zt​)=q(⋅∣zt​).

To operationalize this, the authors define the posterior matching residual r(zt,λ)r(z_t, \lambda)r(zt​,λ) as the difference between the student's and teacher's tilted means. The Posterior Matching Loss minimizes the expected squared residual over noise levels, noisy inputs, and tilts:

LPMD(θ)=Et,zt,λ[∥r(zt,λ)∥22]\mathcal{L}_{\text{PMD}}(\theta) = \mathbb{E}_{t, z_t, \lambda} \left[ \| r(z_t, \lambda) \|_2^2 \right]LPMD​(θ)=Et,zt​,λ​[∥r(zt​,λ)∥22​]

This objective matches a property of the student's conditional distribution rather than forcing individual predictions to equal the teacher's mean.

During training, PMD is trained on noisy states encountered during the student's own sampling process. At each iteration, a training noise level ttt is selected, and ztz_tzt​ is obtained through backward simulation from Gaussian noise zTz_TzT​, with preceding steps detached from the gradient graph. Given ztz_tzt​, a tilt vector λ\lambdaλ is sampled, and the frozen teacher is evaluated at the shifted input zt+ρtλz_t + \rho_t \lambdazt​+ρt​λ to obtain the teacher's tilted posterior mean. The student generates MMM independent predictions x^(m)=Gθ(zt,u(m),t)\hat{x}^{(m)} = G_\theta(z_t, u^{(m)}, t)x^(m)=Gθ​(zt​,u(m),t). The student's tilted mean is estimated using weighted Monte Carlo samples, where weights are derived from a softmax over the dot products λ⊤x^(m)\lambda^\top \hat{x}^{(m)}λ⊤x^(m). The final student objective augments the PMD loss with an auxiliary non-saturating logistic GAN loss:

Lstudent=LPMD+γGANLGAN\mathcal{L}_{\text{student}} = \mathcal{L}_{\text{PMD}} + \gamma_{\text{GAN}} \mathcal{L}_{\text{GAN}}Lstudent​=LPMD​+γGAN​LGAN​

where the discriminator operates on caption-conditioned features extracted by the frozen teacher.

Finally, the authors design PMD to support any-step sampling without training separate students for different step counts. Because PMD learns the conditional posterior of clean images at each noise level rather than explicit transitions between noise levels, the student can predict a clean image from the current noisy state. If further steps are required, this prediction is re-noised to the next selected level using the Gaussian corruption schedule, and the student is evaluated again.

Experiment

The experiments evaluate Posterior Matching Distillation on SDXL and FLUX.2 4B text-to-image models, primarily with four sampling steps, against several few-step baselines and an undistilled reference. Qualitative comparisons show PMD generates more varied subject appearance, scene composition, and backgrounds while preserving prompt semantics, and it supports diverse sampling at different inference step counts. The results also indicate PMD achieves strong distributional quality and prompt adherence, and intra-prompt diversity experiments consistently favor PMD across feature-space, perceptual, and Vendi metrics. Ablations further show that coarse tilt sampling improves diversity and that the stochastic student draw budget affects diversity metrics during training.

On FLUX.2 [klein] 4B, PMD achieves the best FID among four-step methods and leads in HPSv3, PickScore, and CLIP-S. The FLUX.2 reference model posts higher GenEval, DPG-Bench, and OneIG-EN than PMD, revealing a trade-off in prompt-alignment metrics. Moment Matching trails the other four-step approaches on most reported quality and fidelity measures. PMD has the best FID and the highest HPSv3, PickScore, and CLIP-S among the four-step FLUX.2 methods shown. The FLUX.2 reference model outperforms PMD in GenEval, DPG-Bench, and OneIG-EN, while PMD improves FID over that reference.

PMD achieves the strongest diversity results on PartiPrompts among the compared methods, with the highest scores on Vendi, LPIPS, DINOv2, and DreamSim. The gains are most pronounced for DINOv2, DreamSim, and Vendi diversity, while LPIPS improves more modestly. Reported comparisons across SDXL and FLUX.2 indicate a consistent advantage in intra-prompt diversity under fixed prompts and noise. PMD records the highest diversity values across all four metrics for SDXL, surpassing LCM-LoRA, SDXL-Turbo, SDXL-Lightning, Flash, PCM, and DMD2. The relative improvement over the strongest baseline is largest for DINOv2 and DreamSim, while LPIPS shows a smaller gain. A similar diversity advantage is reported for FLUX.2, where PMD improves over baselines across all metrics.

The experiments evaluate four-step text-to-image methods on FLUX.2 and SDXL using quality, prompt-alignment, and diversity benchmarks. PMD achieves the best FID and top HPSv3, PickScore, and CLIP-S among four-step FLUX.2 methods, while the FLUX.2 reference model scores higher on GenEval, DPG-Bench, and OneIG-EN, indicating a trade-off in prompt-alignment metrics; Moment Matching generally trails. On PartiPrompts, PMD consistently yields the highest intra-prompt diversity across Vendi, LPIPS, DINOv2, and DreamSim for both SDXL and FLUX.2, with the largest gains in DINOv2, DreamSim, and Vendi and more modest LPIPS improvement.


بناء الذكاء الاصطناعي بالذكاء الاصطناعي

من الفكرة إلى الإطلاق — سرّع تطوير الذكاء الاصطناعي الخاص بك مع المساعدة البرمجية المجانية بالذكاء الاصطناعي، وبيئة جاهزة للاستخدام، وأفضل أسعار لوحدات معالجة الرسومات.

البرمجة التعاونية باستخدام الذكاء الاصطناعي
وحدات GPU جاهزة للعمل
أفضل الأسعار

HyperAI Newsletters

اشترك في آخر تحديثاتنا
سنرسل لك أحدث التحديثات الأسبوعية إلى بريدك الإلكتروني في الساعة التاسعة من صباح كل يوم اثنين
مدعوم بواسطة MailChimp