---
title: "Paper: TIDE Corrects Teacher-Student Mismatch to Improve LLM On-Policy Distillation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.09836v1"
permalink: "https://feed.astralor.com/items/itm_8b0be48be56da3e9efdd4211"
published: "2026-08-10T16:53:14.000Z"
score: 78
channel: "ai"
tags: ["On-policy Distillation","Qwen3","TIDE","Knowledge Distillation"]
---

# Paper: TIDE Corrects Teacher-Student Mismatch to Improve LLM On-Policy Distillation

## AI commentary

> The degenerate-agreement diagnosis is the real highlight; it warns that token agreement can be gamed, so alignment and distillation metrics deserve closer scrutiny. For teams doing LLM post-training, TIDE's mismatch taxonomy and correction recipes are directly actionable.

## At a glance

Recent work reveals a failure mode in on-policy distillation (OPD) for LLM post-training: degenerate agreement, where students use repetitive loops to achieve near-perfect token agreement with the teacher while producing globally flawed responses. The authors reframe the problem as teacher-student mismatch and categorize mismatch tokens into student-excess and student-deficit types. They propose TIDE (Token-level Independent Deficit-Excess correction), which uses bounded Hellinger shaping to suppress harmful excess tokens and analytic teacher top-K injection to restore deficit probability mass without sampling. Experiments on mathematical reasoning benchmarks with multiple Qwen3 teacher-student pairs show consistent gains over standard OPD and recent baselines. Under strong mismatch, TIDE improves Avg@8 from 6.9% to 20.3%, reduces average response length by 3.6x, and greatly lowers formatting failures. Code is released.

## Key points

- Identifies degenerate agreement in OPD, where students game token agreement through repetitive loops while failing globally.
- Categorizes teacher-student mismatch tokens into student-excess and student-deficit, with distinct harms.
- Proposes TIDE with bounded Hellinger shaping and teacher top-K injection for both mismatch directions.
- Outperforms standard OPD and baselines on Qwen3 math benchmarks; Avg@8 jumps from 6.9% to 20.3% under strong mismatch.

## Full text

On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens are generated by the student but assigned near-zero probability by the teacher; their log-ratio corrections grow unbounded and destabilize the update. Student-deficit tokens, in contrast, are preferred by the teacher but rarely sampled by the student; their absence blocks the transfer of the teacher's reasoning patterns. To tackle these mismatch directions, we propose TIDE (Token-level Independent Deficit-Excess correction), which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass without requiring deficit tokens to be sampled. Across mathematical reasoning benchmarks with multiple Qwen3 teacher-student pairs, TIDE consistently outperforms standard OPD and recent token-selection and reward-shaping baselines. Moreover, the gains of TIDE are more pronounced under strong teacher-student mismatch, where it improves Avg@8 from 6.9% to 20.3%, reduces average response length by a factor of 3.6, and substantially reduces formatting failures. Code is available at https://github.com/yzc-666/TIDE

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_8b0be48be56da3e9efdd4211