---
title: "选择性状态空间中状态使用的精确测量工具及其揭示的输入驱动迁移现象"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11796v1"
permalink: "https://feed.astralor.com/items/itm_03dd3e6d98ba381b9a94a9ac"
published: "2026-07-13T16:48:03.000Z"
score: 78
channel: "ai"
tags: ["Falcon-Mamba","Mamba","模态剪枝","状态分解","输入驱动迁移","选择性状态空间模型"]
---

# 选择性状态空间中状态使用的精确测量工具及其揭示的输入驱动迁移现象

## AI 点评

> 这篇论文非常值得关注，尤其对于研究状态空间模型（如Mamba）推理效率和内部机制的研究者。它首次给出了精确的模态使用测量，并发现了输入驱动迁移这一重要现象，不仅加深了我们对选择性机制的理解，还提供了实用的剪枝方案。对于大模型部署工程师来说，半预算下性能持平的结果极具吸引力。

## 速览

本文提出了一种精确测量选择性状态空间模型（如Mamba）中状态模态使用情况的工具。由于状态矩阵是对角化的，每个通道的输出可以精确分解为各模态的贡献，通过Gram张量可离线计算任意模态子集移除后的输出误差。该工具在Mamba-1系列上验证的精度达2.3×10⁻⁷，并预测剪枝误差的中位数相对偏差仅为5×10⁻⁷。应用该工具于Mamba-1、Falcon-Mamba 7B及Mamba-2，发现模型会根据输入动态重新分配状态空间：携带信号的模态随上下文迁移，在最受影响的层，每输入动态选择模态集可使输出误差减半。归因分析表明，迁移主要由输入依赖的写入映射B_t驱动，而非时间步长。基于该测量的输入调度模态剪枝在不同规模模型上均优于静态剪枝，在半状态预算下性能与未剪枝模型持平。

## 关键点

- 提出了一种精确的状态模态使用测量工具，基于对角状态矩阵实现输出分解。
- 验证了Mamba-1、Mamba-2和Falcon-Mamba系列模型中的输入驱动状态迁移现象。
- 归因分析将迁移主要归因于输入依赖的写入映射B_t，而非时间步长。
- 基于该测量的输入调度剪枝在半状态预算下性能与未剪枝模型持平。

## 全文

Selective state-space models such as Mamba route information through a bank of first-order modes whose input coupling is set by a learned selection mechanism. We give an exact instrument for measuring how a trained model uses these modes. Because the state matrix is diagonal, each channel's output decomposes exactly into per-mode contributions, and a per-(layer, channel, window) Gram tensor yields the exact output error of dropping any subset of modes, offline, at any budget. Validated against the reference implementation to a relative error of $2.3\times10^{-7}$ on the Mamba-1 family where it is exact, the instrument predicts a layer's deployed pruning error to a median relative deviation of $5\times10^{-7}$ over $4{,}464$ configurations, its floor set by the reconstruction. Applying the instrument across the Mamba-1 family (130M--2.8B), the deployed 7B Falcon-Mamba, and Mamba-2, we find that trained models re-allocate their state space with the input: which modes carry the signal migrates across contexts, and at the most affected layers a per-input oracle roughly halves the output error of a fixed mode set. Frozen-signal counterfactuals attribute the migration primarily to the input-dependent write map $B_t$; the timestep usually identified with selectivity carries almost none of it. Input-scheduled mode pruning on this measurement outperforms static, Hankel-based, and layer-adaptive rankings at every scale from 130M to the deployed 7B Falcon-Mamba, and at half the state budget it matches the unpruned model. Because the scheduler reads each window's mode usage from a first pass, this demonstrates realizable headroom; we claim no deployed compute or memory saving.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_03dd3e6d98ba381b9a94a9ac