---
title: "Muon优化器在矩阵分解中学习平衡解：避免慢速鞍点动力学"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30509v1"
permalink: "https://feed.astralor.com/items/itm_2a4d044739f70c37a2253f29"
published: "2026-06-29T16:17:38.000Z"
score: 57
channel: "ai"
tags: ["Muon优化器","学习率调度","矩阵分解","鞍点动力学"]
---

# Muon优化器在矩阵分解中学习平衡解：避免慢速鞍点动力学

## 速览

本文研究Muon优化器在矩阵分解问题中的参数轨迹，发现其与梯度下降有三项主要差异：避免从小初始化开始的慢速鞍点动力学，以相同速率学习所有顶部模式；学习率可超过局部损失尖锐阈值，实现指数退火收敛；守恒量不同但都能从零初始化中找到平衡解。研究还揭示了权重早期自发对齐的速率，并基于Muon结构性质构造出两步达到近完美对齐的学习率调度。

## 关键点

- Muon从零初始化同时学习所有主要模式，避免梯度下降的慢速鞍点序列
- 学习率可突破局部曲率限制，摆脱条件数束缚，支持指数退火加速
- Muon流守恒√(P^T P) - √(Q^T Q)，而非标准梯度流的P^T P - Q^T Q
- 基于Muon结构设计出两步优化调度，实现近完美对齐

## 全文

Matrix factorization (i.e., problems of the form $\min_{\mathbf{P},\mathbf{Q}} \|\mathbf{M}^\star - \mathbf{P}^\top\mathbf{Q}\|_\mathrm{F}^2$) is a minimal learning problem that exhibits both nonlinear parameter dynamics and representation learning. In this setting, we study how parameter trajectories under the Muon optimizer differ from those of gradient descent. We identify three main dynamical differences: 1) Muon avoids the slow saddle-to-saddle dynamics from small initialization. Muon instead learns all the top modes of $\mathbf{M}^\star$ at the same rate, with the smaller modes converging first. 2) Muon remains stable even when the learning rate exceeds the critical threshold set by the local loss sharpness. This frees the learning rate from the condition number of the problem, enabling rapid convergence via exponential learning rate annealing. 3) Once the weights are aligned with each other and the target, Muon flow conserves the matrix quantity $\sqrt{\mathbf{P}^\top \mathbf{P}}-\sqrt{\mathbf{Q}^\top \mathbf{Q}}$, while gradient flow is known to conserve the matrix $\mathbf{P}^\top\mathbf{P} - \mathbf{Q}^\top\mathbf{Q}$. Despite having distinct conserved quantities, both optimizers find the so-called \textit{balanced} solution from vanishing initialization. When training from small random initialization, the weights spontaneously align early in training. We derive the alignment rates in simple settings and show that they predict the empirical alignment rates in general. Finally, we exploit structural properties of Muon to construct a learning rate schedule that achieves near-perfect alignment in only two optimization steps.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_2a4d044739f70c37a2253f29