---
title: "KV Cache Compression Through the Lens of Transform Coding"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.14191v1"
permalink: "https://feed.astralor.com/items/itm_84889d4162a0bc7a6cdd9bd6"
published: "2026-08-14T11:08:01.000Z"
score: 78
channel: "tech"
tags: ["AATC","arXiv","KV cache","Transform Coding","Quantization Compression","Long-context Reasoning"]
---

# KV Cache Compression Through the Lens of Transform Coding

## AI commentary

> This paper's value lies in moving beyond simple quantization error to design compression based on attention propagation properties. For engineers working on long-context inference and LLM deployment, AATC could become a key tool in memory-constrained scenarios, and its engineering and scalability deserve attention.

## At a glance

The key-value (KV) cache is a major memory bottleneck in long-context inference. Existing quantization methods focus on cache reconstruction error but ignore how that error propagates through attention. This paper introduces Attention-Aware Transform Coding (AATC), built on transform coding and reverse water-filling, to allocate bits over a calibration set to minimize attention-aware distortion. Evaluated on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct across LongBench, RULER, GSM8K, MMLU-Pro, and MATH-500, AATC achieves near-lossless accuracy at roughly 5.8x compression, while every baseline degrades in at least some settings.

## Key points

- Theoretical proof: Under a white-noise quantization model, the expected attention-aware distortion decomposes into additive key and value contributions that factor across tokens and channels.
- AATC method: Combines transform coding and reverse water-filling to allocate bits over a calibration set, directly minimizing attention-aware distortion.
- Empirical results: On Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, AATC maintains near-lossless accuracy at about 5.8x compression, outperforming baselines.
- Benchmarks: Validated across LongBench, RULER, GSM8K, MMLU-Pro, MATH-500 for diverse task coverage.

## Full text

The key-value (KV) cache stores information from past tokens and is a major memory bottleneck in long-context inference. Existing quantization methods address this bottleneck by representing the KV cache uniformly with lower-precision data types and designing quantization schemes to minimize reconstruction error in the cache itself, without accounting for how that error propagates through attention mechanisms. We prove that, under a white-noise quantization model, the expected attention-aware distortion decomposes into additive key and value contributions that factor across tokens and channels. Building on transform coding and reverse water-filling, which are classical tools from signal processing and rate-distortion theory, we introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion. On Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, evaluated across LongBench, RULER, GSM8K, MMLU-Pro, and MATH-500, our method achieves near-lossless accuracy at approximately $5.8\times$ compression, whereas each baseline degrades in at least some settings.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_84889d4162a0bc7a6cdd9bd6