---
title: "面向大型音频语言模型声学感知的编码器侧神经元识别与放大"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11801v1"
permalink: "https://feed.astralor.com/items/itm_6f0bada0632454586edea81f"
published: "2026-07-13T16:53:08.000Z"
score: 81
channel: "ai"
tags: ["Audio-Flamingo-3","IAAN","Kimi-Audio","Qwen2.5-Omni","推理时干预","音频编码器"]
---

# 面向大型音频语言模型声学感知的编码器侧神经元识别与放大

## AI 点评

> 这篇论文让人眼前一亮：它没有随大流去堆算力或改架构，而是巧妙利用编码器内部神经元的激活差异，实现了立竿见影的声学感知提升。对LALM开发者和音频研究者特别有参考价值。

## 速览

大型音频语言模型（LALMs）在细粒度非语义语音属性（如说话者情绪）上表现不佳。本文提出IAAN方法，无需训练或标签，通过对比真实波形与噪声参考的激活值，对音频编码器中前馈神经元进行评分，并在推理时放大高分神经元。在十个非语义属性上，该方法在Audio-Flamingo-3、Qwen2.5-Omni和Kimi-Audio上分别提升平均准确率25.7、21.4和9.7个百分点。实验表明，编码器侧和神经元级选择性均不可或缺，而解码侧或语言模型内干预效果甚微。该工作开辟了通过神经元级访问编码器来改善声学感知的推理时新方向。

## 关键点

- IAAN 是首个在音频编码器内部进行神经元级推理时干预的方法，无需重新训练。
- 通过对比真实波形与噪声参考的激活值，识别出对声学感知关键的神经元。
- 在多个主流 LALM 上，IAAN 显著提升了非语义语音属性的准确率，最高提升 25.7 个百分点。
- 验证了编码器侧和神经元级选择性的必要性，解码侧或语言模型内干预效果有限。

## 全文

Large audio-language models (LALMs) often underperform on fine-grained, non-semantic attributes of speech, such as a speaker's emotion, despite strong performance on speech content. Improving this without the cost of retraining calls for an effective inference-time intervention, yet most existing methods intervene only after the audio encoder and operate at a relatively coarse granularity. The encoder itself, where acoustic information is first extracted from the waveform, remains largely unexplored, especially at the level of individual neurons. We introduce IAAN, Identifying and Amplifying Acoustic Neurons, a training-free and label-free method that scores each feed-forward neuron in the audio encoder by contrasting its activation on the real waveform with that on a noise reference lacking the real audio's acoustic information. IAAN then amplifies a small set of the highest-scoring neurons at inference. Across ten non-semantic speech attributes, IAAN improves average accuracy by 25.7 points on Audio-Flamingo-3, 21.4 on Qwen2.5-Omni, and 9.7 on Kimi-Audio. It also improves a model already explicitly fine-tuned to prioritize acoustic evidence. In controlled comparisons, both the encoder locus and neuron-level selectivity prove necessary for this gain. Intervening after the encoder, at the decoding side or inside the language model, yields little to no improvement, or even deteriorates accuracy. The improvement also depends on which specific neurons are amplified, not merely on their number, confirming that IAAN's acoustic score succeeds in identifying the neurons that matter. These results show that a small, precisely targeted intervention inside the audio encoder is an effective and largely untapped way to strengthen the acoustic understanding of LALMs, opening a new direction for inference-time methods that improve acoustic perception through neuron-level access to the encoder.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_6f0bada0632454586edea81f