---
title: "Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.40574v1"
permalink: "https://feed.astralor.com/items/itm_617a1e0fd916d7dd066b0118"
published: "2026-10-29T17:44:42.000Z"
score: 55
channel: "tech"
tags: ["LLM","RAG","多模态","Every Token Needs","Its Own Teacher","Multi-teacher Adaptive On-policy","Self-distillation On-policy Self-distillation","OPSD"]
---

# Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation

## 速览

On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references. However, existing approaches keep the teacher fixed either …

## 关键点

- On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references.
- However, existing approaches keep the teacher fixed either throughout training by consistently exposing it to the full reference, or within each response by adaptively varying partial reference exposure only across resp…
- In this paper, we argue that neither of them is sufficient.

## 全文

On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references. However, existing approaches keep the teacher fixed either throughout training by consistently exposing it to the full reference, or within each response by adaptively varying partial reference exposure only across responses. In this paper, we argue that neither of them is sufficient. Instead, every token needs its own teacher, as the teacher preference can vary token by token within the same response. Through a teacher-intervention pilot study, we validate that the teacher preference differs across tokens, and reveal that a preferred teacher can be identified for each token through majority voting and KL-based selection. Motivated by these findings, we propose Multi-teacher Adaptive On-policy Self-Distillation (MTA-OPSD), the first token-level adaptive OPSD method. Our method constructs a multi-teacher pool consisting of candidate teachers, and enables every token to adaptively select its preferred teacher during training. Experiments with Qwen3 models on mathematical reasoning tasks show that MTA-OPSD consistently outperforms existing OPSD approaches with an average gain of up to $+5.9$ over the base model, highlighting the importance of adapting teacher supervision at the token level. The code is available at https://github.com/xingzhejun/MTA-OPSD.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_617a1e0fd916d7dd066b0118