---
title: "Unified Audio Intelligence Model Audex: Audio-Text LLM Without Regressing on Text Intelligence"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.05196v1"
permalink: "https://feed.astralor.com/items/itm_c25eb067dd18bfcb0ac86963"
published: "2026-07-06T15:11:57.000Z"
score: 77
channel: "tech"
tags: ["Audex","MoE","Nemotron-Labs","Speech Understanding","Audio-Text Unification"]
---

# Unified Audio Intelligence Model Audex: Audio-Text LLM Without Regressing on Text Intelligence

## AI commentary

> This paper is noteworthy for demonstrating how to preserve the powerful capabilities of a text-only model within a unified multimodal framework, which is critical for building general AI agents. For researchers and engineers working on speech interaction, multimodal LLMs, and AI infrastructure, Audex's architecture and training methods provide a reproducible reference. Particularly striking is its achievement of SOTA audio performance without degrading text intelligence, addressing a core challenge in multimodal learning.

## At a glance

Nemotron-Labs introduces Audex, a unified audio-text LLM built on a 30B-parameter MoE text LLM, using a single Transformer decoder that fuses audio inputs into the text embedding space and treats text tokens and quantized audio output tokens uniformly. Trained on meticulously curated datasets of 157.4B audio tokens and 320.5B text tokens through multi-stage supervised training, Cascade RL, and multi-domain on-policy distillation, Audex achieves SOTA in audio understanding, speech recognition/translation, TTS, audio generation, and speech-to-speech generation, while preserving the backbone’s reasoning, alignment, knowledge, long-context, and agentic capabilities with minimal regression. Model checkpoints are open-sourced.

## Key points

- Architecture: Single Transformer decoder with unified audio-text processing, compatible with standard LLM training/inference.
- Training: 157.4B audio tokens + 320.5B text tokens, multi-stage supervised training, text Cascade RL, multi-domain on-policy distillation.
- Performance: SOTA on audio understanding/generation tasks; text capabilities preserved with marginal or no regression.

## Full text

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_c25eb067dd18bfcb0ac86963