---
title: "Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26001v1"
permalink: "https://feed.astralor.com/items/itm_0ada9936f1daee42f8a1526d"
published: "2026-07-28T17:18:11.000Z"
score: 73
channel: "ai"
tags: ["Muon optimizer","SAM","Generalization Robustness","Matrix-Aware Optimization","Spectral Perturbation"]
---

# Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm

## AI commentary

> What makes this paper stand out is its principled unification of SAM and Muon from a geometric perspective, offering deep insights for optimizer and generalization theory researchers. For practitioners, the spectral perturbation + Muon combination could become a practical trick for training Vision Transformers and similar models.

## At a glance

This paper reexamines Sharpness-Aware Minimization (SAM) from a geometric perspective, introducing layerwise spectral inner perturbations for matrix-valued hidden-layer parameters and combining them with AdamW/SGDW or Muon for outer updates. Experiments on ImageNet-1K with ViT-Small/16 and ResNet-50 show that the combination of a spectral inner step with a Muon outer step consistently achieves the best validation accuracy, highlighting the importance of matrix-aware geometry in both stages of SAM. This work offers new insights into optimizer-generalization understanding and advances practical training methods.

## Key points

- Proposes layerwise spectral inner perturbations that leverage matrix structure to replace traditional vector perturbations, enhancing SAM's adaptability to parameter geometry.
- Introduces Muon optimizer into the outer update stage of SAM, synergizing with spectral inner steps to achieve best validation accuracy on both ViT and ResNet.
- Systematic experiments on ImageNet-1K reveal the advantage of matrix-aware geometry over scalar or vector geometries in SAM.

## Full text

Sharpness-Aware Minimization (SAM) aims to improve generalization by encouraging insensitivity to small, worst-case parameter perturbations. However, the notion of a "small" perturbation is inherently geometry-dependent: while existing SAM variants have explored a wide range of choices, a clear perspective on which geometries are most effective in practice remains elusive. Recent work on matrix-aware optimization, particularly the Muon optimizer, suggests that respecting the matrix structure of hidden-layer weights can lead to strong empirical performance. Motivated by this, we study matrix-aware geometry in both stages of SAM: we introduce a layerwise spectral inner perturbation for matrix-valued hidden-layer parameters and combine it with either AdamW/SGDW or Muon in the outer update. Across ImageNet-1K experiments on ViT-Small/16 and ResNet-50, we find that the combination of a spectral inner step with a Muon outer step performs consistently strongly, achieving the best validation accuracy on both models among the evaluated methods.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_0ada9936f1daee42f8a1526d