---
title: "JoLT：基于联合 Tucker 分解与 JL 残差分配的近无损 KV 缓存压缩方法"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.12550v1"
permalink: "https://feed.astralor.com/items/itm_870cc2b28619f2746b0241eb"
published: "2026-07-14T09:23:20.000Z"
score: 79
channel: "tech"
tags: ["JoLT","KV Cache 压缩","Meta","Mistral","大模型推理优化","张量分解"]
---

# JoLT：基于联合 Tucker 分解与 JL 残差分配的近无损 KV 缓存压缩方法

## AI 点评

> 这篇论文值得所有关注大模型推理效率的工程师和研究者仔细阅读。它把张量分解这样的经典数学工具用在了当前最头疼的KV缓存膨胀问题上，效果出奇地好。JoLT的核心洞察——缓存天然就是三阶张量，而之前大家却只切二维片去压缩——启发性很强。对部署长上下文模型（比如128K、1M token）的团队来说，这个2-3倍近无损压缩加上FlashJoLT的加速，意味着能直接省下一半以上的显存，或者把上下文推得更长。我特别欣赏它用统一拉格朗日对偶来分配秩和位宽的思路，让压缩策略能自适应不同层和键值差异。

## 速览

大模型推理中 KV 缓存内存占比随上下文长度急剧增长。现有低秩与量化方法未利用缓存的三阶张量结构。本文提出 JoLT，对缓存的 token 和特征轴实施部分 Tucker 分解，并用 Johnson-Lindenstrauss 旋转低比特残差恢复截断能量。通过拉格朗日对偶统一分配 Tucker 秩与残差位宽，实现 2-3 倍近无损压缩：在 Mistral-7B 和 LLaMA-2-13B 上困惑度、GSM8K 准确率、RULER 检索均保持在统计噪声内。Frobenius 误差较跨层 SVD 和 4-bit 量化降低一个数量级。随机 SVD 变体 FlashJoLT 带来 5-13 倍压缩加速。

## 关键点

- 首次将 KV 缓存视为三阶张量（头、token、特征轴），对 token 与特征轴进行部分 Tucker 分解
- 使用 JL 旋转低比特残差恢复截断能量，联合分配 Tucker 秩与残差位宽
- 在 Mistral-7B 和 LLaMA-2-13B 上实现 2-3 倍近无损压缩，性能无退化
- FlashJoLT 变体通过随机 SVD 将压缩速度提升 5-13 倍

## 全文

The key-value (KV) cache has become the dominant memory cost of transformer inference. It grows with batch size, context length, and depth, and at long context it, rather than the model weights, sets the ceiling on throughput. Two families of methods reduce it. Low-rank methods factor two-dimensional slices of the cache, either per-head matrices or cross-layer feature blocks, and quantization methods lower the bit-width of every entry. Neither family exploits the fact that the cache at a layer is naturally a third-order tensor whose three axes, the heads, the tokens, and the features, carry very different amounts of redundancy. We take this tensor view directly. Our method, JoLT, applies a partial Tucker decomposition that compresses only the token and feature axes while leaving the head and layer axes intact, and then restores the energy that truncation discards with a Johnson-Lindenstrauss (JL) rotated low-bit residual. A single Lagrangian dual allocates the Tucker ranks and the residual bit-widths together, per layer group and separately for keys and values, under one byte budget. The result is a near-lossless 2-3x compression: perplexity, GSM8K accuracy, and RULER needle-in-a-haystack retrieval all stay at or within statistical noise of the uncompressed baseline on both a grouped-query-attention model (Mistral-7B-v0.3) and a multi-head-attention model (LLaMA-2-13B). At 2x, JoLT reconstructs the cache to relative Frobenius error 0.009 (K) and 0.006 (V) on both architectures, roughly an order of magnitude below cross-layer SVD and 4-bit quantization. A randomized-SVD variant, FlashJoLT, delivers a 5-13x compression-time speedup at matched quality.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_870cc2b28619f2746b0241eb