---
title: "FreqDepthKV：频率引导的深度共享实现长上下文LLM高效KV缓存压缩"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06519v1"
permalink: "https://feed.astralor.com/items/itm_1c0d08aef0e09c21cb0d7363"
published: "2026-07-07T17:26:28.000Z"
score: 77
channel: "ai"
tags: ["FreqDepthKV","KV缓存压缩","大型语言模型","长上下文推理"]
---

# FreqDepthKV：频率引导的深度共享实现长上下文LLM高效KV缓存压缩

## AI 点评

> 这篇论文针对长上下文LLM推理中KV缓存的内存瓶颈，提出了一种基于频率分解的自适应压缩策略。我认为其在线探针按注意力头动态选择压缩模式的设计相当精巧，能在极小开销下实现接近全缓存的准确率。对于从事LLM推理优化和模型部署的研究者与工程师，本文提供了一种即插即用的有效方案，特别适合需要处理极长上下文的任务。

## 速览

FreqDepthKV是一种推理时KV缓存压缩方法，通过将相邻层的KV状态分解为共享低频深度分量和稀疏高频残差，并利用在线探针按注意力头动态分配压缩模式。在长上下文问答、检索、摘要和代码生成任务中，FreqDepthKV在极小缓存预算下保持准确率：32k预填充窗口下达到58.3 Exact Match、63.0 F1、32.5 ROUGE-L、48.1 pass@1，解码吞吐量70.4 tokens/s，首令牌时延2.06秒，峰值KV内存6.2GB，实现3.9倍有效压缩比，接近全KV性能且显著优于此前压缩方案。

## 关键点

- 提出频率引导的深度共享范式，将KV缓存拆分为共享低频分量和残差高频分量。
- 设计轻量在线探针，按注意力头对重建敏感度动态选择共享深度模式、残差深度模式或精确缓存模式。
- 在多项长上下文基准上，仅需较小缓存即可保持准确率，解码吞吐量达70.4 tokens/s，压缩比3.9倍。

## 全文

Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1c0d08aef0e09c21cb0d7363