---
title: "Bidirectional Centered Score Matching for Discrete Diffusion Distillation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.04279v1"
permalink: "https://feed.astralor.com/items/itm_94f0bca9bfef187281b66ea2"
published: "2026-11-03T04:05:41.000Z"
score: 56
channel: "tech"
tags: ["推理效率","安全","多模态","Bidirectional Centered Score","Matching","Discrete Diffusion Distillation","Discrete","Although"]
---

# Bidirectional Centered Score Matching for Discrete Diffusion Distillation

## 速览

Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives. Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optim…

## 关键点

- Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives.
- Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optimization geometry and teacher-only candidate truncation may provide insufficient supervision under agg…
- Concrete Score Matching (CSM) provides an alternative within-position objective by aligning relative logit geometry, but its complete pairwise form incurs $\mathcal{O}(|\mathcal{V}|^2)$ complexity.

## 全文

Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives. Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optimization geometry and teacher-only candidate truncation may provide insufficient supervision under aggressive few-step distillation. Concrete Score Matching (CSM) provides an alternative within-position objective by aligning relative logit geometry, but its complete pairwise form incurs $\mathcal{O}(|\mathcal{V}|^2)$ complexity. We propose Bidirectional Centered Score Matching (BCSM), an efficient within-position distillation objective that constructs an $\mathcal{O}(K)$ bidirectional active set, reduces pairwise comparisons to $\mathcal{O}(K^2)$, and factorizes them into an $\mathcal{O}(K)$ objective. We further introduce cross-timestep transition alignment. Experiments demonstrate strong quality--efficiency trade-offs, including up to $6\times$ NFE reduction with a 16.9\% relative HumanEval gain and a 15.1\% relative MMMU gain at single-step inference.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_94f0bca9bfef187281b66ea2