---
title: "HyFlash: A Prefix-Memory Prior for On-Device Speculative Decoding"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.07263v1"
permalink: "https://feed.astralor.com/items/itm_7fda16d0be6453e31ca8499c"
published: "2026-11-05T00:31:21.000Z"
score: 56
channel: "ai"
tags: ["RAG","HyFlash","Prefix-Memory Prior","On-Device Speculative Decoding","Parallel","Under","KV-cache"]
---

# HyFlash: A Prefix-Memory Prior for On-Device Speculative Decoding

## At a glance

Parallel block drafting accelerates speculative decoding by predicting multiple future positions in a single backbone pass, but on-device deployment constrains both drafter capacity and context storage. Under tight memo…

## Key points

- Parallel block drafting accelerates speculative decoding by predicting multiple future positions in a single backbone pass, but on-device deployment constrains both drafter capacity and context storage.
- Under tight memory budgets, drafters must model long-range dependencies and fine-grained draft-position interactions without the growing KV-cache overhead of full-context attention.
- We propose HyFlash, a compact parallel drafter that introduces a prefix-memory prior to separate long-range context modeling from fine-grained draft-position interactions with limited capacity.

## Full text

Parallel block drafting accelerates speculative decoding by predicting multiple future positions in a single backbone pass, but on-device deployment constrains both drafter capacity and context storage. Under tight memory budgets, drafters must model long-range dependencies and fine-grained draft-position interactions without the growing KV-cache overhead of full-context attention. We propose HyFlash, a compact parallel drafter that introduces a prefix-memory prior to separate long-range context modeling from fine-grained draft-position interactions with limited capacity. Local attention captures recent context and draft-position interactions, while recurrent memory compresses historical target features into fixed-size states. This inductive bias enables effective local and long-range modeling with bounded draft-side memory. Experiments show that HyFlash achieves higher acceptance than full-context attention with substantially less KV storage and with larger gains for shallow drafters and long contexts.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_7fda16d0be6453e31ca8499c