---
title: "Fusion Training for Mathematical Generalization in Large Language Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.09893v1"
permalink: "https://feed.astralor.com/items/itm_32d845f2839b58e46c62ddfc"
published: "2026-08-10T17:41:19.000Z"
score: 77
channel: "ai"
tags: ["Fusion Bench","Thinking Mode Fusion","Large Language Models","Mathematical Reasoning","Training Method"]
---

# Fusion Training for Mathematical Generalization in Large Language Models

## AI commentary

> This paper is especially valuable for those training reasoning models—it quantifies the cost of switching between thinking modes rather than staying conceptual. More non-thinking data degrades deep reasoning, a tension rarely demonstrated with data before. Notably, they open-sourced Fusion Bench, which will make future comparisons of scheduling strategies much easier.

## At a glance

This paper systematically studies Thinking Mode Fusion (TMF) training, which enables a single large language model to support both concise responses and long-form reasoning. The authors analyze the effects of data ratio and training schedule between thinking and non-thinking modes on mathematical problem solving, revealing an asymmetric interaction: increasing non-thinking supervision reduces thinking-mode accuracy. Different training schedules modulate this trade-off, and the optimal schedule depends on the data ratio. The study quantifies a negative correlation between the two modes, highlighting inherent tension and offering practical guidance for effective TMF training.

## Key points

- Proposes a fusion training framework uniting non-thinking and thinking modes in one model
- Finds asymmetric interaction and negative correlation between thinking and non-thinking supervision
- Shows optimal training schedule depends on data ratio, guiding TMF configuration
- Releases code and data (Fusion Bench) to support further research

## Full text

Thinking Mode Fusion (TMF) enables large language models to support both concise responses and long-form reasoning by unifying a non-thinking mode and a thinking mode within a single model. However, its training dynamics, including the \emph{data ratio} and \emph{training schedule} between the two modes, remain underexplored. In this work, we present a systematic study of TMF by analyzing the effects of the training schedule and data ratio between thinking and non-thinking modes. Focusing on mathematical problem solving, we construct a benchmark with multiple thinking-to-non-thinking data ratios and three training schedules. Our results reveal an asymmetric interaction between the two modes: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. We further show that different training schedules modulate this trade-off and that the optimal schedule depends on the data ratio. Finally, we quantify a negative correlation between non-thinking and thinking mode supervision, highlighting an inherent tension between these two modes. These findings provide practical guidance for designing effective TMF training settings. All code and data are released to support further research at: \href{https://github.com/caocongfeng/Fusion-Bench.git}{\textbf{Fusion Bench}}.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_32d845f2839b58e46c62ddfc