---
title: "RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.06347v1"
permalink: "https://feed.astralor.com/items/itm_659c18bd05dc97fa4f74b7b8"
published: "2026-08-06T17:52:06.000Z"
score: 72
channel: "tech"
tags: ["RP-OPSD","On-Policy Self-Distillation","Multilingual Reasoning Transfer","Mathematical Reasoning"]
---

# RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer

## AI commentary

> Multilingual reasoning transfer has long been constrained by undifferentiated supervision signals. RP-OPSD cleverly uses teacher-view distribution shifts to automatically locate reasoning pivots. For researchers working on multilingual LLM training or distillation, this paper offers a simple and reproducible improvement strategy worth reading.

## At a glance

RP-OPSD proposes a self-distillation method for multilingual reasoning transfer. On-policy self-distillation (OPSD) provides dense token-level supervision but does not explicitly prioritize the reasoning signals that matter most for cross-lingual transfer. The method decomposes target-language reasoning into surface text and reasoning pivots, and uses the distributional shift between matched teacher views with and without an English reference solution as a proxy for pivots, enabling privileged distillation and reference anchoring. Experiments on mathematical reasoning benchmarks spanning 17 languages and multiple difficulty levels show that RP-OPSD outperforms strong multilingual reasoning baselines and OPSD variants. Analysis shows it concentrates privileged distillation on reasoning-control and problem-conditioned state-update tokens, downweighting surface realization tokens. Code is open-sourced.

## Key points

- Introduces reasoning pivots as key supervision signals for cross-lingual transfer, defined as tokens that advance or redirect reasoning. 
- Locates reasoning pivots unsupervised via distributional shift between teacher views with and without English reference solutions. 
- Outperforms OPSD and strong baselines on mathematical reasoning benchmarks across 17 languages. 
- Open-sources code for easy reproduction and extension.

## Full text

Multilingual reasoning transfer is crucial for extending reasoning capabilities of large language models (LLMs) beyond high-resource languages. On-policy self-distillation (OPSD) and its variants have emerged as a promising paradigm, providing dense token-level supervision on student-generated rollouts, yet their objectives do not explicitly prioritize reasoning signals most critical to cross-lingual transfer. We characterize that target-language reasoning comprises the generation of both surface text and reasoning pivots, which are decisions that advance or redirect the reasoning process and shape subsequent inference. This motivates concentrating privileged distillation around such pivots. We therefore propose RP-OPSD, Reasoning-Pivot-guided On-Policy Self-Distillation, using the distributional shift between matched teacher views with and without an English reference solution as an operational proxy to guide privileged distillation and reference anchoring. Experiments on mathematical reasoning benchmarks covering 17 languages and multiple difficulty levels show that our method outperforms strong multilingual reasoning baselines and OPSD variants. Further analysis reveals that RP-OPSD concentrates privileged distillation on reasoning-control and problem-condistioned state-update tokens, while downweighting it for tokens that mainly support surface realization. Our code is available at https://github.com/NJUNLP/RP-OPSD.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_659c18bd05dc97fa4f74b7b8