---
title: "A Better Critic Matters for Distribution Matching Distillation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.42135v1"
permalink: "https://feed.astralor.com/items/itm_65b00dd3e4a03872cdae02a1"
published: "2026-10-30T17:58:26.000Z"
score: 53
channel: "ai"
tags: ["多模态","Better Critic Matters","Distribution Matching Distillation","DMD","SiT","ImageNet"]
---

# A Better Critic Matters for Distribution Matching Distillation

## 速览

Distribution Matching Distillation (DMD) trains a few-step student using the difference between a teacher's score and an online critic's estimate of the student distribution's score. We investigate what the critic needs…

## 关键点

- Distribution Matching Distillation (DMD) trains a few-step student using the difference between a teacher's score and an online critic's estimate of the student distribution's score.
- We investigate what the critic needs, in capacity and in information, to provide a reliable training signal.
- On the capacity side, controlled experiments across seven SiT model sizes on ImageNet show that increasing teacher size while keeping a small critic fixed can degrade student quality and cause training collapse.

## 全文

Distribution Matching Distillation (DMD) trains a few-step student using the difference between a teacher's score and an online critic's estimate of the student distribution's score. We investigate what the critic needs, in capacity and in information, to provide a reliable training signal. On the capacity side, controlled experiments across seven SiT model sizes on ImageNet show that increasing teacher size while keeping a small critic fixed can degrade student quality and cause training collapse. Scaling the critic alongside the teacher, a strategy we call joint scaling, prevents collapse in our experiments and allows the student to benefit from larger teachers. On the information side, rather than forcing the critic to model a step-agnostic mixture of student outputs, we introduce a step-aware critic (STAC) that conditions on the student's generation step to estimate step-specific scores. Together, joint scaling and STAC consistently improve few-step distillation across class-conditional ImageNet generation, text-to-image generation with SDXL and FLUX.2, and text-to-video generation with Wan2.1.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_65b00dd3e4a03872cdae02a1