---
title: "Accelerating Long-Context Decoding in Large-Scale Hybrid LLMs Without Retraining"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.06133v1"
permalink: "https://feed.astralor.com/items/itm_35c5c4a0f2187874b2602bde"
published: "2026-11-04T08:18:38.000Z"
score: 57
channel: "tech"
tags: ["LLM","Accelerating Long-Context Decoding","Large-Scale Hybrid LLMs","Without Retraining The","LLMs","Hybrid"]
---

# Accelerating Long-Context Decoding in Large-Scale Hybrid LLMs Without Retraining

## 速览

The full-attention layers of large language models (LLMs) make every generated token more expensive than the previous one, slowing models down on long-context tasks. Hybrid models reduce this cost by replacing most full…

## 关键点

- The full-attention layers of large language models (LLMs) make every generated token more expensive than the previous one, slowing models down on long-context tasks.
- Hybrid models reduce this cost by replacing most full-attention layers with linear-attention layers, but the first still dominate decoding at long contexts.
- We accelerate these remaining layers without any training or weight changes using combined sparse attention: each layer attends to a sliding window of recent tokens, a pinned prefix, and a fixed budget of most relevant …

## 全文

The full-attention layers of large language models (LLMs) make every generated token more expensive than the previous one, slowing models down on long-context tasks. Hybrid models reduce this cost by replacing most full-attention layers with linear-attention layers, but the first still dominate decoding at long contexts. We accelerate these remaining layers without any training or weight changes using combined sparse attention: each layer attends to a sliding window of recent tokens, a pinned prefix, and a fixed budget of most relevant tokens in the context. The window size and budget are found on a per-layer basis using a multi-objective search algorithm. Across six hybrid models from three families, from 4B to 397B parameters, the selected configurations preserve their baseline model's accuracy on 21 long-context datasets. Decoding 64k/256k tokens becomes up to 1.26x/2.22x faster on Qwen3.5-4B, 1.20x/2.23x on Qwen3.5-122B-A10B, and 1.16x/2.06x on Ornith-1.0-397B.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_35c5c4a0f2187874b2602bde