---
title: "AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.26004v1"
permalink: "https://feed.astralor.com/items/itm_be18dedbe13e4be53fe4f353"
published: "2026-08-26T16:50:02.000Z"
score: 72
channel: "ai"
tags: ["AsymSpec","δ-fusion","Divergence-aware Acceptance Gate","Speculative Decoding","Agentic LLMs"]
---

# AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs

## AI commentary

> This paper is worth attention for breaking the identical-context assumption of speculative decoding with an asymmetric design that preserves both accuracy and speed. For teams working on agent inference optimization and model deployment, this approach may become a standard option in compression scenarios.

## At a glance

AsymSpec is a context-asymmetric speculative decoding framework proposed to address the accuracy drop caused by context compression in agentic LLM pipelines. Traditional speculative decoding requires the drafter and verifier to share identical context. AsymSpec breaks this symmetry by letting a lightweight drafter read the full input while the large verifier operates on a compressed view. A contrastive δ-fusion of logits and a divergence-aware acceptance gate maintain verification stability. Evaluated across four agentic capabilities and two end-to-end benchmarks, AsymSpec achieves ~90% of full-context accuracy on average, with 1.3-1.7x throughput speedups at 0.2-0.3x compute cost.

## Key points

- Breaks the identical-context assumption of speculative decoding by having the drafter read full input and the verifier use compressed context.
- Uses contrastive δ-fusion and a divergence-aware acceptance gate to maintain verification stability and improve draft acceptance.
- Reaches ~90% full-context accuracy on average with 1.3-1.7x throughput speedups at 0.2-0.3x compute cost.
- Validated across isolated text capabilities and end-to-end agent benchmarks.

## Full text

Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive $δ$-fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end agent benchmarks, AsymSpec reaches $\approx 90\%$ of full-context accuracy on average, delivering $1.3$--$1.7\times$ throughput speedups at $0.2$--$0.3\times$ the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_be18dedbe13e4be53fe4f353