---
title: "利用指令微调与模型合并实现推理模型适应"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.14895v1"
permalink: "https://feed.astralor.com/items/itm_639e985548b682454df56d8e"
published: "2026-07-16T12:11:25.000Z"
score: 77
channel: "tech"
tags: ["成本效益","指令微调","推理语言模型","模型合并","监督微调"]
---

# 利用指令微调与模型合并实现推理模型适应

## AI 点评

> 这篇论文让我眼前一亮——用不到3美元就解决了推理模型在非数学/编程领域的适应问题，简直是“白菜价”的高效方案。对于初创团队或预算有限的研究人员来说，这种即插即用式的方法极大降低了门槛。如果你正在为RLM落地发愁，这篇绝对值得细读。

## 速览

推理语言模型在数学和编程等可验证领域表现出色，但在缺乏可靠验证器的领域训练仍具挑战。本文提出一种低成本方法：先对推理模型进行经典指令微调（无需推理轨迹），再将其与原始推理模型合并，从而恢复并提升目标领域推理能力。实验表明，该方法在编程、文本摘要等可验证和难验证领域均显著提升性能，且成本低于3美元，同时保留模型在其他领域的能力。

## 关键点

- 提出指令微调+模型合并的推理模型适应方法，无需推理轨迹。
- 利用现有监督微调数据，避免复杂的强化学习流程。
- 在编程和文本摘要等任务上验证有效性，成本低于3美元。
- 不损害模型在其他领域的原有能力。

## 全文

Reasoning language models (RLMs) have demonstrated impressive performance in domains such as mathematics and coding. These domains permit reliable verification of model outputs, which is important for enabling the reinforcement learning that drives RLM performance gains. However, training RLMs on domains that lack reliable verifiers remains challenging. Meanwhile, for both verifiable and unverifiable domains, large amounts of unused supervised fine-tuning data with human-written solutions exist. In this work, we show that these data can be used efficiently to further improve RLM performance. For this, we first use classic instruction tuning, supervised fine-tuning without reasoning traces, on the RLM. Next, we merge our instruction-tuned model with the original reasoning model, recovering its reasoning behavior on the target domain. Our extensive evaluation demonstrates that our technique improves RLM performance in both verifiable and hard-to-verify domains, including coding and text summarization, while preserving RLM capabilities across other domains. Importantly, our method is highly cost-effective, enabling such improvements for less than USD $3.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_639e985548b682454df56d8e