---
title: "WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06461v1"
permalink: "https://feed.astralor.com/items/itm_c26babd2aa9659b188e56ae3"
published: "2026-07-07T16:22:59.000Z"
score: 75
channel: "tech"
tags: ["LLM","Text-to-Speech","WordVoice","WordVoice-5A","Word-level control","Prosodic planning"]
---

# WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS

## AI commentary

> This paper is noteworthy because it directly addresses a critical pain point for practical LLM-TTS: fine-grained control. For developers and researchers working on speech synthesis, audiobook production, and video dubbing, WordVoice's decoupled word-level control enables precise adjustment of speed, stress, and emotion without sacrificing naturalness. The editor believes this method has strong potential for industrial adoption, especially combined with LLM's language understanding capabilities, promising more natural prosodic adaptation in the future.

## At a glance

Recent LLM-based TTS systems achieve remarkable naturalness but rely on implicit end-to-end generation, resulting in coarse-grained control. In scenarios demanding precise stylistic interventions and strict temporal alignment, such as audiobook narration and video dubbing, the inability to explicitly manipulate word-level acoustic attributes remains a critical bottleneck. This paper proposes WordVoice, a unified framework: first, it constructs WordVoice-5A, a 4.7k-hour bilingual dataset with five-dimensional word-level annotations (duration, boundary, energy, pitch, and tone) developed through a linguistically-guided pipeline; second, it introduces a bound-token mechanism to transform implicit generation into an explicit controllable paradigm, enabling adaptive multi-task prosodic planning and flexible manual intervention; third, it augments the token-to-waveform stage with a fine-grained acoustic modulation module to bridge the resolution gap between highly compressed discrete tokens and continuous waveforms. Experiments show WordVoice achieves superior decoupled control over multiple acoustic dimensions while maintaining competitive zero-shot synthesis stability. Code and audio samples are publicly available.

## Key points

- Constructs WordVoice-5A: a 4.7k-hour bilingual dataset with five-dimensional word-level acoustic annotations.
- Proposes a bound-token mechanism for explicit acoustic planning, enabling multi-task prosody control and manual intervention.
- Introduces a fine-grained acoustic modulation module to align word-level attributes between discrete tokens and continuous waveforms.
- Experiments demonstrate superior decoupled control across multiple acoustic dimensions while maintaining competitive zero-shot synthesis stability.

## Full text

While recent Large Language Model (LLM)-based Text-to-Speech (TTS) systems have achieved remarkable naturalness, they predominantly rely on implicit end-to-end generation paradigms, resulting in coarse-grained control. In scenarios demanding precise stylistic interventions and strict temporal alignment, such as audiobook narration and video dubbing, the inability to explicitly manipulate word-level acoustic attributes remains a critical bottleneck. This limitation is primarily amplified by the severe scarcity of fine-grained annotated datasets and the architectural challenge of integrating multi-dimensional control signals into discrete autoregressive generation. To address this, we propose a unified framework for highly precise word-level control. First, we construct WordVoice-5A, a massive 4.7k-hour bilingual dataset featuring five-dimensional word-level annotations (duration, boundary, energy, pitch and tone) developed through a rigorous linguistically-guided pipeline. Second, we introduce WordVoice to transform the implicit generation process into an explicit, highly controllable paradigm. Specifically, we introduce a bound-token mechanism within the LLM to formulate an explicit ``acoustic planning'' process, enabling adaptive multi-task prosodic planning and flexible manual intervention. Furthermore, we augment the token-to-waveform stage with a fine-grained acoustic modulation module, bridging the resolution gap to strictly align word-level attributes between highly compressed discrete tokens and continuous waveforms. Extensive experiments demonstrate that WordVoice achieves superior, decoupled control over multiple acoustic dimensions while maintaining competitive zero-shot synthesis stability. The code and audio samples are publicly available at https://xxh333.github.io/wordvoice-demo/.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_c26babd2aa9659b188e56ae3