---
title: "Prefix Sliding: A New Method for Efficient Test-Time Scaling"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.26070v1"
permalink: "https://feed.astralor.com/items/itm_6dca582753d4de676e34c3ff"
published: "2026-08-26T17:37:15.000Z"
score: 71
channel: "ai"
tags: ["arXiv","Prefix Sliding","Large Language Models","Reinforcement Learning","Test-time Scaling"]
---

# Prefix Sliding: A New Method for Efficient Test-Time Scaling

## AI commentary

> What's most striking is the counter-intuitive finding that most intermediate reasoning tokens are unimportant, offering a lightweight fix for long-reasoning costs. Teams working on inference optimization or deploying long-thinking models can try Prefix Sliding almost out-of-the-box; it's worth immediate experimentation.

## At a glance

Test-time scaling uses extra compute to improve performance, but full attention over long reasoning traces incurs prohibitive memory costs. We find that most intermediate reasoning tokens lose importance as the model continues. We propose Prefix Sliding, which discards tokens not in the prefix or the recent window of a few thousand tokens, capping memory regardless of reasoning length. Without training, it makes existing models 3x faster while maintaining performance. With reinforcement learning training, it enables scaling to reasoning traces beyond 100k tokens and achieves better performance. Ablations show it outperforms summarizing intermediate tokens or vanilla sliding window.

## Key points

- Finds that most intermediate tokens lose importance during reasoning, justifying selective discarding.
- Prefix Sliding makes existing models 3x faster without training while maintaining performance.
- With RL training, it enables reasoning traces beyond 100k tokens for better performance.
- Ablations show it outperforms intermediate summarization and vanilla sliding window.

## Full text

Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window. Our code is at https://github.com/Muennighoff/prefix-sliding

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_6dca582753d4de676e34c3ff