---
title: "Softmax Routing Dynamics in Mixture-of-Experts Models: Collapse and Recovery"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.39442v1"
permalink: "https://feed.astralor.com/items/itm_5f39cc88b046a9eb17e14986"
published: "2026-10-29T06:16:04.000Z"
score: 54
channel: "ai"
tags: ["Softmax Routing Dynamics","Mixture-of-Experts Models","Collapse","Recovery Softmax","Softmax","For","Under","Integrations"]
---

# Softmax Routing Dynamics in Mixture-of-Experts Models: Collapse and Recovery

## 速览

Softmax routing in mixture-of-experts models can concentrate probability on a subset of experts, suppressing access to the others. We study how this concentration forms, why it persists, and how suppressed probabilities…

## 关键点

- Softmax routing in mixture-of-experts models can concentrate probability on a subset of experts, suppressing access to the others.
- We study how this concentration forms, why it persists, and how suppressed probabilities recover.
- For fixed experts, a stationary-loss barrier identifies the face supporting the optimal routed output.

## 全文

Softmax routing in mixture-of-experts models can concentrate probability on a subset of experts, suppressing access to the others. We study how this concentration forms, why it persists, and how suppressed probabilities recover. For fixed experts, a stationary-loss barrier identifies the face supporting the optimal routed output. In a common-target square-loss model with freely trainable expert outputs, slow adaptation tracks fixed-expert dynamics on finite horizons, while every fixed positive adaptation rate yields an interior routing limit. An exact normal-velocity decomposition separates complementary-expert covariance from relative gradient advantage. Under positive two-sided power bounds along recovery paths, uniform over initial masses, the resulting coefficient's order of vanishing gives logarithmic or power-law recovery times. Integrations of constructed fields reproduce these orders. An exact discrete log-odds identity relates neural updates to threshold crossing. In paired Transformer-MoE training, removing a class-conditional router-gradient contrast delays confirmed recovery and reduces accumulated log-odds on fixed tokens, both with and without capacity constraints.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_5f39cc88b046a9eb17e14986