---
title: "LLM缓存该用哪种逐出策略？跨工作负载、容量与编码器的系统研究"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.20280v1"
permalink: "https://feed.astralor.com/items/itm_33c9b48905ee7183414c224a"
published: "2026-08-20T17:14:19.000Z"
score: 77
channel: "ai"
tags: ["arXiv","CLEVER","LFU","LLM语义缓存","MiniLM","缓存逐出策略"]
---

# LLM缓存该用哪种逐出策略？跨工作负载、容量与编码器的系统研究

## AI 点评

> 这篇论文最大的价值在于给了一个反直觉但非常实用的结论：别折腾复杂逐出策略，先检查你的缓存命中到底能不能用。对做LLM推理优化和基础设施的工程师来说，它提醒你质量调整后的命中率才是真正的KPI，也给了LFU一个非常扎实的默认选择。

## 速览

该论文首次在统一协议下系统评估了FIFO、LRU、LFU、ARC、GDSF、流式SISO及语义冗余策略在LLM语义缓存中的表现。基于CLEVER框架，研究覆盖三类查询语料、三种缓存容量和两种编码器，结果发现没有任何策略在18种设置中比LFU提升超过0.041个百分点；但在小容量下FIFO和流式SISO最高落后8.67和8.55个百分点。作者用条件填充结果解释了为何几何感知策略难以获得额外收益，并进一步指出在MiniLM中位近邻阈值下，仅2.1%-3.9%的命中可真正替代答案，质量调整后命中率从51%-60%骤降至1.1%-2.2%。研究建议先验证答案有效性，再用精确搜索测试策略差异。

## 关键点

- 在18种设置中，没有任何逐出策略比LFU提升超过0.041个百分点，LFU是最强简单默认策略。
- 小容量下FIFO和流式SISO比LFU最多落后8.67和8.55个百分点，替换策略并非无关紧要。
- 精确查找与未命中插入条件下，新条目无法在命中半径内获得邻近条目，几何感知策略难以获得额外冗余信号。
- MiniLM中位近邻阈值下仅2.1%-3.9%的命中可真正替代答案，原始命中率51%-60%经质量调整后降至1.1%-2.2%。

## 全文

Semantic caches reuse an LLM response when the incoming query embedding lies near a cached query, but proposed eviction policies have rarely been compared under one protocol. Using CLEVER, we evaluate FIFO, LRU, LFU, ARC, GDSF, a single-pass streaming adaptation of SISO, and a semantic-redundancy policy across three ordered, deduplicated query corpora, three cache capacities, and two encoders. No evaluated policy improves on LFU by more than 0.041 percentage points in any of the eighteen settings. Replacement is not irrelevant: FIFO and streaming SISO trail LFU by as much as 8.67 and 8.55 points, respectively, at tight capacity. We explain the missing upside with a conditional packing result. Under exact lookup and insert-on-miss, a newly inserted entry cannot have a resident neighbor within the hit radius, so a geometry-aware eviction rule receives little new redundancy signal. A separate audit exposes a larger problem with the evaluated operating point. At MiniLM's median nearest-neighbor threshold, only 2.1-3.9% of sampled LMSYS and QQP hits are judged answer-substitutable, reducing raw hit rates of 51-60% to quality-adjusted rates of 1.1-2.2%. The cross-encoder study further shows that thresholds do not transfer between embedding models. LFU is the strongest simple default in this protocol; deployment decisions should first establish answer validity and then test sub-point policy differences with exact search.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_33c9b48905ee7183414c224a