---
title: "Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.14895v1"
permalink: "https://feed.astralor.com/items/itm_639e985548b682454df56d8e"
published: "2026-07-16T12:11:25.000Z"
score: 77
channel: "tech"
tags: ["Cost-Effective","Instruction Tuning","Reasoning Language Model","Model Merging","Supervised Fine-Tuning"]
---

# Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

## AI commentary

> This paper caught my eye — solving RLM adaptation in non-math/coding domains for under $3 is a 'peanuts' high-efficiency solution. For startups or resource-limited researchers, this plug-and-play approach dramatically lowers the barrier. If you're struggling with RLM deployment, this is a must-read.

## At a glance

Reasoning Language Models excel in verifiable domains like math and coding, but struggle in domains lacking reliable verifiers. This paper proposes a low-cost method: first perform classic instruction tuning (without reasoning traces) on the RLM, then merge it with the original RLM to recover and enhance reasoning behavior on the target domain. Experiments show significant improvements in both verifiable and hard-to-verify domains (e.g., coding, text summarization) for under $3, while preserving capabilities elsewhere.

## Key points

- Proposes instruction tuning + model merging for RLM adaptation, without requiring reasoning traces.
- Leverages existing supervised fine-tuning data, avoiding complex reinforcement learning pipelines.
- Validated on coding and text summarization tasks, costing less than $3.
- Preserves model capabilities across other domains.

## Full text

Reasoning language models (RLMs) have demonstrated impressive performance in domains such as mathematics and coding. These domains permit reliable verification of model outputs, which is important for enabling the reinforcement learning that drives RLM performance gains. However, training RLMs on domains that lack reliable verifiers remains challenging. Meanwhile, for both verifiable and unverifiable domains, large amounts of unused supervised fine-tuning data with human-written solutions exist. In this work, we show that these data can be used efficiently to further improve RLM performance. For this, we first use classic instruction tuning, supervised fine-tuning without reasoning traces, on the RLM. Next, we merge our instruction-tuned model with the original reasoning model, recovering its reasoning behavior on the target domain. Our extensive evaluation demonstrates that our technique improves RLM performance in both verifiable and hard-to-verify domains, including coding and text summarization, while preserving RLM capabilities across other domains. Importantly, our method is highly cost-effective, enabling such improvements for less than USD $3.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_639e985548b682454df56d8e