---
title: "FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06519v1"
permalink: "https://feed.astralor.com/items/itm_1c0d08aef0e09c21cb0d7363"
published: "2026-07-07T17:26:28.000Z"
score: 77
channel: "ai"
tags: ["FreqDepthKV","KV-cache compression","Large Language Models","Long-context Reasoning"]
---

# FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

## AI commentary

> This paper tackles the memory bottleneck of KV caches in long-context LLM inference with an adaptive compression strategy based on frequency decomposition. I find the online probe that dynamically selects compression modes per attention head to be elegant, achieving near-full-cache accuracy with minimal overhead. For researchers and engineers working on LLM inference optimization and deployment, this work offers a plug-and-play solution especially valuable for extremely long-context tasks.

## At a glance

FreqDepthKV is an inference-time KV cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to different compression modes adaptively. On long-context QA, retrieval, summarization, and code generation benchmarks, it achieves 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1 under a 32k-token prefill, with 70.4 tokens/s decoding throughput, 2.06s TTFT, and 6.2 GB peak KV memory—a 3.9× effective compression ratio, matching full KV performance and outperforming prior compressed-cache methods.

## Key points

- Introduces a frequency-guided depth sharing paradigm that decomposes KV cache into shared low-frequency components and residual high-frequency components.
- Designs a lightweight online probe that dynamically selects shared-depth, residual-depth, or exact cache modes per attention head based on reconstruction sensitivity.
- Achieves competitive accuracy on multiple long-context benchmarks with significantly reduced cache; decoding throughput reaches 70.4 tokens/s with a 3.9× compression ratio.

## Full text

Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_1c0d08aef0e09c21cb0d7363