---
title: "OmniDelta：面向全模态大语言模型的技能驱动Token压缩预算分配"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.25669v1"
permalink: "https://feed.astralor.com/items/itm_9020d75c6eb01083fa7c3589"
published: "2026-07-28T12:50:31.000Z"
score: 81
channel: "ai"
tags: ["OmniDelta","Qwen2.5-Omni","Token压缩","全模态大语言模型","技能驱动预算分配"]
---

# OmniDelta：面向全模态大语言模型的技能驱动Token压缩预算分配

## AI 点评

> 这篇论文切中了全模态大模型落地的一个痛点：长序列token开销。OmniDelta无需额外训练，仅通过巧妙设计预算分配就能在保持精度的前提下显著加速，对部署全模态模型的工程团队极具参考价值。尤其值得关注的是其“技能池”思路，将查询意图融入分配，比单纯基于注意力的方法更智能。

## 速览

全模态大语言模型（OmniLLMs）需处理文本、音频、视频的联合理解，但长音频-视频token序列带来巨大内存和推理开销。现有压缩方法仅关注固定预算下的重要token选择，而忽略预算分配问题。论文提出OmniDelta，一种无需训练、技能驱动的框架，通过构建音频和视频技能池实现意图感知的跨模态预算分配与内容感知的模态内预算分配。在Qwen2.5-Omni-7B上，25% token保留率下GPU内存减少22%，端到端加速1.64倍，在精度-效率帕累托前沿上建立新基准。

## 关键点

- 首次系统研究全模态大语言模型中的token预算分配问题，发现直接查询-音频/视频相似度不可靠
- 提出OmniDelta框架，无需训练，通过构建技能池实现意图感知的跨模态预算分配和内容感知的模态内分配
- 在Qwen2.5-Omni-7B上25% token保留率下GPU内存减少22%，端到端加速1.64倍，超越现有方法
- 方法可兼容现有修剪策略，保持总保留token比例不变，仅改变预算分配位置

## 全文

Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs. Existing compression methods mainly focus on selecting important tokens under fixed budgets, leaving the preceding budget-allocation problem underexplored. We show that direct query-to-audio/video similarity is unreliable for inter-modal budget allocation, and that uniform intra-modal budgets can miss key evidence while retaining redundant content. To address these limitations, we propose OmniDelta, a training-free, skill-driven framework that couples intent-aware inter-modal allocation with content-aware intra-modal allocation. OmniDelta first constructs audio and video skill pools to shift the fixed retained-token budget according to query demand, then reallocates modality budgets over audio segments and video frames using local complexity and temporal redundancy. The resulting local budgets can be combined with existing pruning strategies, preserving the total retained-token ratio while changing where the budget is spent. Experiments on four audio-video benchmarks with two Qwen2.5-Omni models show that OmniDelta establishes a new accuracy-efficiency Pareto frontier across pruning ratios. At 25% token retention on Qwen2.5-Omni-7B, OmniDelta reduces GPU memory by 22.0% and achieves a 1.64x end-to-end speedup over full-token inference.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_9020d75c6eb01083fa7c3589