---
title: "An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Reveals"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11796v1"
permalink: "https://feed.astralor.com/items/itm_03dd3e6d98ba381b9a94a9ac"
published: "2026-07-13T16:48:03.000Z"
score: 78
channel: "ai"
tags: ["Falcon-Mamba","Mamba","Mode Pruning","State Decomposition","Input-Driven Migration","Selective State-Space Models"]
---

# An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Reveals

## AI commentary

> This paper is highly recommended for researchers studying inference efficiency and internal mechanisms of state-space models like Mamba. It provides the first exact measurement of mode usage and reveals the important phenomenon of input-driven migration, deepening our understanding of selection mechanisms while offering practical pruning strategies. The result of matching unpruned performance at half the state budget is especially attractive for deployment engineers.

## At a glance

This paper presents an exact instrument for measuring how trained selective state-space models (such as Mamba) use their state modes. Thanks to the diagonal state matrix, each channel's output decomposes exactly into per-mode contributions, and a Gram tensor enables offline computation of the output error when dropping any subset of modes. Validated on Mamba-1 with relative error 2.3×10⁻⁷, the instrument predicts pruning error with median relative deviation 5×10⁻⁷ across 4,464 configurations. Applied to Mamba-1, Falcon-Mamba 7B, and Mamba-2, it reveals that models reallocate state space dynamically based on input: the modes carrying the signal migrate across contexts, and at the most affected layers a per-input oracle roughly halves the output error of a fixed mode set. Counterfactual analysis attributes migration primarily to the input-dependent write map B_t, not the timestep. Input-scheduled mode pruning based on this measurement outperforms static, Hankel-based, and layer-adaptive rankings at all scales from 130M to 7B, and at half the state budget matches the unpruned model.

## Key points

- An exact instrument for measuring state mode usage is proposed, leveraging diagonal state matrix for output decomposition.
- Input-driven state migration is verified across Mamba-1, Mamba-2, and Falcon-Mamba series.
- Counterfactual analysis attributes migration primarily to the input-dependent write map B_t, not the timestep.
- Input-scheduled pruning based on this measurement matches unpruned model performance at half the state budget.

## Full text

Selective state-space models such as Mamba route information through a bank of first-order modes whose input coupling is set by a learned selection mechanism. We give an exact instrument for measuring how a trained model uses these modes. Because the state matrix is diagonal, each channel's output decomposes exactly into per-mode contributions, and a per-(layer, channel, window) Gram tensor yields the exact output error of dropping any subset of modes, offline, at any budget. Validated against the reference implementation to a relative error of $2.3\times10^{-7}$ on the Mamba-1 family where it is exact, the instrument predicts a layer's deployed pruning error to a median relative deviation of $5\times10^{-7}$ over $4{,}464$ configurations, its floor set by the reconstruction. Applying the instrument across the Mamba-1 family (130M--2.8B), the deployed 7B Falcon-Mamba, and Mamba-2, we find that trained models re-allocate their state space with the input: which modes carry the signal migrates across contexts, and at the most affected layers a per-input oracle roughly halves the output error of a fixed mode set. Frozen-signal counterfactuals attribute the migration primarily to the input-dependent write map $B_t$; the timestep usually identified with selectivity carries almost none of it. Input-scheduled mode pruning on this measurement outperforms static, Hankel-based, and layer-adaptive rankings at every scale from 130M to the deployed 7B Falcon-Mamba, and at half the state budget it matches the unpruned model. Because the scheduler reads each window's mode usage from a first pass, this demonstrates realizable headroom; we claim no deployed compute or memory saving.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_03dd3e6d98ba381b9a94a9ac