---
title: "vToken：面向可回收 KV 缓存的 Token 级虚拟化方案"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.13263v1"
permalink: "https://feed.astralor.com/items/itm_97958df04623aa4beb7dc596"
published: "2026-08-13T14:01:21.000Z"
score: 77
channel: "tech"
tags: ["KV 缓存","LLM 推理优化","PagedAttention","vLLM","vToken"]
---

# vToken：面向可回收 KV 缓存的 Token 级虚拟化方案

## AI 点评

> 这篇论文最值得关注的是用虚拟化思路化解粒度错配，而不是单纯堆新缓存算法。对做推理服务底层优化的人，它给出了一个低侵入、可复用的抽象层，可能会影响后续 KV 管理器设计。

## 速览

大模型推理中 KV 缓存随序列长度和批大小增长，成为关键内存瓶颈。PagedAttention 以固定大小块管理内存，而现有逐出算法按 token 粒度操作，导致块内碎片化，大量已分配内存无法回收。vToken 提出轻量级 token 级虚拟化层，通过 token 表间接寻址解耦逻辑生命周期与物理块放置，并异步重打包实现物理回收，兼容 PagedAttention 内核与 CUDA Graph。在 vLLM 中实现后，相比 Naive-Evict 基线，每请求保留 KV 块减少 27.2% 至 72.3%，SLA 约束吞吐最高提升 1.37 倍，最大并发扩展 2 倍，且集成成本从 500 多行降至 50 行以内。

## 关键点

- vToken 通过 token 表间接寻址，将逻辑 token 存活状态与物理块放置解耦，解决块内碎片导致的内存不可回收问题。
- 异步重打包活动 token，实现物理内存回收，同时保持 PagedAttention 内核与 CUDA Graph 兼容。
- 在 vLLM 中集成 H2O、Random、Scissorhands 等逐出策略，每请求保留 KV 块减少 27.2%–72.3%，SLA 约束吞吐最高提升 1.37 倍。
- 策略集成代码量从 500 多行降至 50 行以内，受限 active-KV 预算下最大并发扩展可达 2 倍。

## 全文

Large language model serving faces a critical memory bottleneck: the KV cache grows with sequence length and batch size. PagedAttention uses fixed-size memory blocks to reduce allocator-level fragmentation, but recent KV eviction algorithms operate at a token granularity finer than block-level management. This mismatch causes intra-block fragmentation, leaving a large fraction of allocated KV memory unreclaimable. We present vToken, a lightweight token-level virtualization layer that decouples logical token liveness from physical block placement. vToken maintains a stable logical token view through token-table indirection and realizes physical reclamation by repacking live tokens asynchronously. The design preserves PagedAttention kernels and CUDA Graph compatibility. We implement vToken in vLLM and evaluate it with H2O, Random, and Scissorhands across models. Compared with a paired Naive-Evict baseline, vToken reduces retained KV blocks per request by 27.2\%--72.3\% and improves SLA-constrained throughput by up to 1.37$\times$. Under a constrained active-KV budget, it extends the maximum feasible concurrency by up to 2$\times$, while reducing the per-policy integration footprint from 500+ lines to under 50.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_97958df04623aa4beb7dc596