---
title: "DepthWeave-KV: 长上下文KV缓存压缩的Token自适应跨层残差分解"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06523v1"
permalink: "https://feed.astralor.com/items/itm_8b0702e3e27b10790db8bc17"
published: "2026-07-07T17:29:01.000Z"
score: 78
channel: "ai"
tags: ["DepthWeave-KV","KV缓存压缩","Token自适应","长上下文推理"]
---

# DepthWeave-KV: 长上下文KV缓存压缩的Token自适应跨层残差分解

## AI 点评

> 这篇论文解决的是长上下文LLM推理中最棘手的KV缓存内存问题。它的亮点在于不再一刀切压缩，而是让模型自己决定哪些token需要保留更多细节，这种token自适应思路既聪明又实用。尤其对做模型部署和推理优化的工程师来说，8.3倍内存节省和免校准特性意味着可以直接集成到生产环境，值得关注其开源实现。

## 速览

深度上下文语言模型推理受限于KV缓存的带宽和容量，现有压缩方法常对层或token使用统一预算，导致检索性能下降。本文提出DepthWeave-KV，一种token自适应缓存压缩方法，通过跨层共享低秩通道基分解key/value状态，同时保留注意力敏感token的轻量残差。结合token条件深度路由器，对指令型和检索关键token分配更高重建秩，并利用注意力输出探针的无校准在线误差追踪动态调整压缩，无需重训练。在LongBench、Needle-in-a-Haystack、L-Eval等基准上，DepthWeave-KV实现近全缓存任务质量，KV内存降低8.3倍，64K上下文下达到72.8 tokens/s。

## 关键点

- 提出DepthWeave-KV，通过跨层共享低秩基分解和token特定残差实现自适应KV缓存压缩，避免统一预算导致的检索退化。
- 引入token条件深度路由器和注意力输出探针的在线误差追踪，实现免校准、动态调整压缩率，无需重训练基础模型。
- Fused CUDA实现联合执行基查找、残差反量化与注意力投影，显著降低解码时内存流量。
- 在多个长上下文基准上达到近全缓存质量，KV内存降低8.3倍，64K上下文吞吐量72.8 tokens/s。

## 全文

Long-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation. We introduce DepthWeave-KV, a token-adaptive cache compression method that factorizes key and value states across neighboring transformer layers using shared low-rank channel bases while retaining lightweight token-specific residuals where attention behavior is sensitive. DepthWeave-KV combines cross-depth residual factorization with a token-conditional depth router that allocates higher reconstruction rank to instruction-bearing and retrieval-critical tokens, and uses calibration-free online error tracking from attention-output probes to adapt compression during generation without retraining the base model. A fused CUDA implementation jointly performs basis lookup, residual dequantization, and attention projection to reduce decode-time memory traffic. Across LongBench, Needle-in-a-Haystack, L-Eval, and long-form QA and summarization benchmarks, DepthWeave-KV achieves near-full-cache task quality with substantially lower memory use, improving average score and retrieval accuracy over prior compressed caches while reaching 8.3x KV memory reduction and 72.8 tokens per second at 64K context.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_8b0702e3e27b10790db8bc17