---
title: "ISO：面向RLVR的原生优化栈——继承谱系，优化框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.19331v1"
permalink: "https://feed.astralor.com/items/itm_7c3927dcc50f7cbc834dd3ad"
published: "2026-07-21T17:51:36.000Z"
score: 79
channel: "ai"
tags: ["AdamW","Muon","Qwen3","RLVR","等谱优化"]
---

# ISO：面向RLVR的原生优化栈——继承谱系，优化框架

## AI 点评

> 这篇论文切中当前大模型后训练的核心痛点——RLVR优化失明。ISO框架的思路简洁优雅（固定谱、只调框架），实验数据也很有说服力，尤其是训练步数压缩到1/3以下。对于从事RLHF/RLVR研究和部署的团队来说，这是可以直接拿来用的解决方案。推荐给所有关注大模型推理能力提升、模型合并与高效微调的研究者和工程师。

## 速览

本文针对强化学习可验证奖励（RLVR）优化层缺失问题，提出基于权重奇异结构的谱继承现象，并设计等谱优化（ISO）框架。离线ISO-Merger无需后训练数据即可合并多个专家能力，实现最强无数据合并性能；在线ISO-Optimizer在固定底模谱系下优化框架变量，在1.5B至8B参数推理、编程任务中显著加速收敛（如Qwen3-8B-Base上ISO-AdamW仅用100步达到AdamW 270步的准确率）。ISO揭示了RLVR后训练应以“继承谱系、优化框架”为核心设计。

## 关键点

- 发现RLVR可以通过谱继承复用底模的权重谱系，仅通过改变输入输出奇异框架获得新行为。
- 提出ISO框架：离线ISO-Merger支持无数据合并多个专家，在线ISO-Optimizer在固定谱下训练框架变量，收敛更快。
- 在Qwen3-8B-Base上，ISO-AdamW用100步达到AdamW 270步的准确率（0.495），并在210步提升至0.509。

## 全文

Reinforcement learning with verifiable rewards (RLVR) is rapidly advancing the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-space updates remains poorly understood. Building on our prior analysis (Zhu et al., 2025), we study this missing layer through the singular structure of model weights and identify spectral inheritance: RLVR can reuse the base model's weight spectra while acquiring new behavior through changes in the associated input and output singular frames. We operationalize spectral inheritance as Isospectral Optimization (ISO), an RLVR-native, fixed-spectrum optimization framework with complementary offline and online instantiations. Offline, ISO-Merger combines the frame changes of shared-base specialists into a single fixed-spectrum model, requiring no post-merge data, rollouts, gradient updates, or on-policy distillation (OPD). It recovers complementary specialist capabilities and achieves the strongest aggregate performance among the compared data-free merging methods. Online, ISO-Optimizer applies a chosen base optimizer, including AdamW and Muon, to the frame variables while keeping the base spectra fixed. Across reasoning and coding tasks ranging from 1.5B to 8B parameters, ISO-Optimizer improves accuracy in the reported runs and reaches matched scores with substantially fewer training steps. On Qwen3-8B-Base, AdamW reaches an aggregate accuracy of 0.495 after 270 training steps. ISO-AdamW reaches the same accuracy after only 100 training steps and improves further to 0.509 after 210 training steps. Together, ISO offers a concrete answer to RLVR's missing optimization layer: rather than inheriting pre-training optimization wholesale, design post-training around the structure of reward-driven adaptation: inherit the spectrum, optimize the frames.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_7c3927dcc50f7cbc834dd3ad