---
title: "RP-OPSD：基于推理支点引导的在线策略自蒸馏，实现多语言推理迁移"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.06347v1"
permalink: "https://feed.astralor.com/items/itm_659c18bd05dc97fa4f74b7b8"
published: "2026-08-06T17:52:06.000Z"
score: 72
channel: "tech"
tags: ["RP-OPSD","在线策略自蒸馏","多语言推理迁移","数学推理"]
---

# RP-OPSD：基于推理支点引导的在线策略自蒸馏，实现多语言推理迁移

## AI 点评

> 多语言推理迁移一直受限于“监督信号不分主次”的窠臼，RP-OPSD 的巧妙之处在于用教师视图的分布偏移自动定位推理支点。对做多语言 LLM 训练或蒸馏的同学来说，这篇论文提供了一个简单且可复现的改进策略，值得细读。

## 速览

RP-OPSD 提出针对多语言推理迁移的自蒸馏方法。在线策略自蒸馏（OPSD）虽提供 token 级监督，但未聚焦跨语言推理最关键的信号。该方法将目标语言推理分解为表面文本与推理支点，利用有无英文参考答案时教师视图的分布偏移作为推理支点的代理，进行特权蒸馏与参考锚定。在覆盖 17 种语言、多难度级别的数学推理基准上，该方法超越强基线与 OPSD 变体，并将特权蒸馏集中于推理控制与问题条件状态更新 token。代码已开源。

## 关键点

- 提出推理支点概念，以决策推进或重定向推理的 token 作为跨语言迁移的关键监督信号。
- 利用有/无英文参考答案的教师视图分布偏移，无监督地定位推理支点。
- 在 17 种语言数学推理基准上超越 OPSD 及强基线，验证有效性。
- 开源代码，便于复现与扩展。

## 全文

Multilingual reasoning transfer is crucial for extending reasoning capabilities of large language models (LLMs) beyond high-resource languages. On-policy self-distillation (OPSD) and its variants have emerged as a promising paradigm, providing dense token-level supervision on student-generated rollouts, yet their objectives do not explicitly prioritize reasoning signals most critical to cross-lingual transfer. We characterize that target-language reasoning comprises the generation of both surface text and reasoning pivots, which are decisions that advance or redirect the reasoning process and shape subsequent inference. This motivates concentrating privileged distillation around such pivots. We therefore propose RP-OPSD, Reasoning-Pivot-guided On-Policy Self-Distillation, using the distributional shift between matched teacher views with and without an English reference solution as an operational proxy to guide privileged distillation and reference anchoring. Experiments on mathematical reasoning benchmarks covering 17 languages and multiple difficulty levels show that our method outperforms strong multilingual reasoning baselines and OPSD variants. Further analysis reveals that RP-OPSD concentrates privileged distillation on reasoning-control and problem-condistioned state-update tokens, while downweighting it for tokens that mainly support surface realization. Our code is available at https://github.com/NJUNLP/RP-OPSD.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_659c18bd05dc97fa4f74b7b8