---
title: "LinQuant: Functional Invariances for Low-Precision Linear Recurrences"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.39418v1"
permalink: "https://feed.astralor.com/items/itm_598c75f703e96681a9cdcaaf"
published: "2026-10-29T05:59:51.000Z"
score: 56
channel: "tech"
tags: ["LLM","推理效率","RAG","LinQuant","Functional Invariances","Low-Precision Linear Recurrences","Linear","However"]
---

# LinQuant: Functional Invariances for Low-Precision Linear Recurrences

## 速览

Linear recurrences are emerging as practical alternatives to softmax attention in large language models, appearing in both standalone and hybrid architectures. However, despite their rapid emergence, low-precision infer…

## 关键点

- Linear recurrences are emerging as practical alternatives to softmax attention in large language models, appearing in both standalone and hybrid architectures.
- However, despite their rapid emergence, low-precision inference remains much less developed for these models than for softmax attention.
- During prefill, chunkwise-parallel algorithms cast much of the recurrence as dense matrix multiplications, suggesting a natural path to low-precision execution.

## 全文

Linear recurrences are emerging as practical alternatives to softmax attention in large language models, appearing in both standalone and hybrid architectures. However, despite their rapid emergence, low-precision inference remains much less developed for these models than for softmax attention. During prefill, chunkwise-parallel algorithms cast much of the recurrence as dense matrix multiplications, suggesting a natural path to low-precision execution. Simply quantizing these operations in their native form, however, can be fragile. In this work, we identify functional invariances that preserve the full-precision computation while making the operands more amenable to quantization. Building on this observation, we introduce LinQuant, a framework for low-precision recurrent prefill based on seven such invariances, including rotations, diagonal rescalings, and alternative placements of recurrent decay and write factors. We derive when these transformations remain valid across common recurrence families spanning linear, gated, and delta-rule updates, and show that they provide complementary degrees of freedom for improving low-precision accuracy. Finally, we translate this framework to production by augmenting SSDi8, an existing low-precision prefill pipeline for Mamba-2, with the applicable invariances. LinQuant improves perplexity at matched latency on Mamba-2 1.3B, 2.7B, and 8B and Nemotron-H 8B, demonstrating the practical utility of invariance-based quantization for recurrent models.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_598c75f703e96681a9cdcaaf