---
title: "统一音频智能模型Audex：不牺牲文本智能的音频-文本大语言模型"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.05196v1"
permalink: "https://feed.astralor.com/items/itm_c25eb067dd18bfcb0ac86963"
published: "2026-07-06T15:11:57.000Z"
score: 77
channel: "tech"
tags: ["Audex","MoE","Nemotron-Labs","语音理解","音频文本统一"]
---

# 统一音频智能模型Audex：不牺牲文本智能的音频-文本大语言模型

## AI 点评

> 这篇论文值得关注，因为它展示了如何在统一多模态框架中保持纯文本模型的强大能力，这对构建通用AI智能体至关重要。对于从事语音交互、多模态LLM和AI基础设施的研究者与工程师，Audex的架构和训练方法提供了可复现的参考。尤其引人注目的是其在不降低文本智能的前提下实现音频SOTA，这解决了多模态学习的核心挑战。

## 速览

Nemotron-Labs推出Audex，基于30B参数MoE文本LLM构建的统一音频文本大语言模型，采用单一Transformer解码器，统一处理音频编码和文本/量化音频输出。通过1574亿音频tokens和3205亿文本tokens的精心策划数据集进行多阶段监督训练，结合文本Cascade RL和多领域on-policy蒸馏。模型在音频理解、语音识别翻译、TTS、音频生成和语音到语音生成方面达到SOTA，同时保持文本LLM的推理、对齐、知识、长上下文和Agent能力几乎无退化。模型权重已开源。

## 关键点

- 模型架构：单一Transformer解码器，统一音频文本处理，与标准LLM训练推理基础设施兼容。
- 训练数据：1574亿音频tokens + 3205亿文本tokens，多阶段监督训练+文本Cascade RL+多领域on-policy蒸馏。
- 性能：音频理解和生成任务SOTA，文本能力几乎无退化。

## 全文

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_c25eb067dd18bfcb0ac86963