---
title: "Pass the Baton: Trajectory-Relayed On-Policy Distillation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26057v1"
permalink: "https://feed.astralor.com/items/itm_d3d3959cb5cfc5ea6d18ef5f"
published: "2026-07-28T17:59:46.000Z"
score: 78
channel: "ai"
tags: ["Qwen3","On-policy Distillation","Mathematical Reasoning"]
---

# Pass the Baton: Trajectory-Relayed On-Policy Distillation

## AI commentary

> This paper is especially valuable for researchers working on model compression and distillation. It hits the pain point of on-policy distillation—prefix error accumulation—and cleverly exploits teacher-student behavior differences as trigger signals without extra labeling. If you are improving small model reasoning capabilities, this offers a low-cost and effective new approach.

## At a glance

This paper proposes Relay-OPD, an improved on-policy distillation method that addresses the prefix failure problem where the student model commits to wrong reasoning directions. By detecting failed prefixes and letting the teacher model briefly take over to produce a 'teacher leg', then resuming student generation for optimization, it effectively corrects erroneous trajectories. On mathematical reasoning benchmarks, the 1.7B model outperforms standard OPD by 5.73% on average, with training trajectory length reduced by over 50%.

## Key points

- Identifies teacher-student continuation asymmetry on failed prefixes: teachers tend to redirect while students continue along the original direction.
- Converts this asymmetry into a label-free handoff trigger for constructing relay trajectories.
- Limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy.
- On 8 mathematical reasoning benchmarks, 1.7B model outperforms standard OPD by 5.73% and the strongest baseline FastOPD by 1.49% on average.

## Full text

On-policy distillation (OPD) grounds token-level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks, Relay-OPD achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_d3d3959cb5cfc5ea6d18ef5f