---
title: "论文：TIDE 矫正师生失配，提升 LLM 策略蒸馏性能"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.09836v1"
permalink: "https://feed.astralor.com/items/itm_8b0be48be56da3e9efdd4211"
published: "2026-08-10T16:53:14.000Z"
score: 78
channel: "ai"
tags: ["On-policy Distillation","Qwen3","TIDE","知识蒸馏"]
---

# 论文：TIDE 矫正师生失配，提升 LLM 策略蒸馏性能

## AI 点评

> 我特别关注“退化一致性”这个发现——学生用循环重复骗过对齐指标，说明纯 token 一致并不可靠。对做 LLM 后训练和对齐的同学，TIDE 的失配分类与矫正思路值得直接借鉴。

## 速览

最新论文揭示 LLM 后训练中 on-policy distillation（OPD）的退化一致性：学生用循环重复换取与教师 token 高度一致，但全局回答错误。作者提出 TIDE 矫正师生失配，以有界 Hellinger 整形抑制学生超额 token，并用教师 top-K 注入补足学生不足 token。在 Qwen3 数学推理上，TIDE 显著优于基线，强失配下 Avg@8 从 6.9% 提升至 20.3%，响应长度减少 3.6 倍。

## 关键点

- 发现 OPD 中的退化一致性：学生通过循环重复换取与教师 token 一致，但整体回答错误。
- 将师生失配 token 分为学生超额和学生不足两类，并分析各自对训练更新的影响。
- 提出 TIDE，用有界 Hellinger 整形抑制超额 token，用教师 top-K 注入恢复不足 token。
- 在 Qwen3 多个师生配对数学基准上超过 OPD 与基线，强失配下 Avg@8 从 6.9% 升至 20.3%。

## 全文

On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens are generated by the student but assigned near-zero probability by the teacher; their log-ratio corrections grow unbounded and destabilize the update. Student-deficit tokens, in contrast, are preferred by the teacher but rarely sampled by the student; their absence blocks the transfer of the teacher's reasoning patterns. To tackle these mismatch directions, we propose TIDE (Token-level Independent Deficit-Excess correction), which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass without requiring deficit tokens to be sampled. Across mathematical reasoning benchmarks with multiple Qwen3 teacher-student pairs, TIDE consistently outperforms standard OPD and recent token-selection and reward-shaping baselines. Moreover, the gains of TIDE are more pronounced under strong teacher-student mismatch, where it improves Avg@8 from 6.9% to 20.3%, reduces average response length by a factor of 3.6, and substantially reduces formatting failures. Code is available at https://github.com/yzc-666/TIDE

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_8b0be48be56da3e9efdd4211