---
title: "融合训练：实现大模型数学推理泛化的新方法"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.09893v1"
permalink: "https://feed.astralor.com/items/itm_32d845f2839b58e46c62ddfc"
published: "2026-08-10T17:41:19.000Z"
score: 77
channel: "ai"
tags: ["Fusion Bench","Thinking Mode Fusion","大型语言模型","数学推理","训练方法"]
---

# 融合训练：实现大模型数学推理泛化的新方法

## AI 点评

> 这篇论文对正在做推理模型训练的同学特别有参考价值——它没有停留在概念上，而是把'思维模式切换'的代价量化了出来。非思维数据一多，深度推理能力就掉，这种张力此前少有人直接给出数据。值得关注的是，他们开源了Fusion Bench，未来对比各种调度策略会更方便。

## 速览

本文系统研究了Thinking Mode Fusion (TMF)训练方法，该方法让大模型在同一模型中同时支持简洁回答和长链推理。作者分析了思维模式与非思维模式之间的数据比例和训练调度对数学问题求解的影响，发现两者存在非对称交互：增加非思维监督会降低思维模式准确性，且不同训练调度可调节这种权衡，最优调度依赖数据比例。研究揭示了两种模式间的负相关关系，为设计有效的TMF训练设置提供了实用指导。

## 关键点

- 提出融合训练框架，统一非思维与思维模式于单一模型
- 发现思维与非思维监督存在非对称交互和负相关
- 最优训练调度依赖数据比例，为TMF设置提供指导
- 开放全部代码与数据（Fusion Bench）支持后续研究

## 全文

Thinking Mode Fusion (TMF) enables large language models to support both concise responses and long-form reasoning by unifying a non-thinking mode and a thinking mode within a single model. However, its training dynamics, including the \emph{data ratio} and \emph{training schedule} between the two modes, remain underexplored. In this work, we present a systematic study of TMF by analyzing the effects of the training schedule and data ratio between thinking and non-thinking modes. Focusing on mathematical problem solving, we construct a benchmark with multiple thinking-to-non-thinking data ratios and three training schedules. Our results reveal an asymmetric interaction between the two modes: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. We further show that different training schedules modulate this trade-off and that the optimal schedule depends on the data ratio. Finally, we quantify a negative correlation between non-thinking and thinking mode supervision, highlighting an inherent tension between these two modes. These findings provide practical guidance for designing effective TMF training settings. All code and data are released to support further research at: \href{https://github.com/caocongfeng/Fusion-Bench.git}{\textbf{Fusion Bench}}.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_32d845f2839b58e46c62ddfc