Qwen-Audio-3.0-TTS: Freely Controllable and Highly Robust Speech Synthesis with Multi-Stage Training Paradigm

2026/07/27 by Bajian Xiang, Cheng Wen, Han Zhao +12 · 1 citation
#eess.AS

paper · pdf

Abstract

In this report, we present Qwen-Audio-3.0-TTS, a production-oriented speech synthesis system that jointly advances content consistency, speaker similarity, prosodic naturalness, audio quality, controllability, multilingual coverage, efficiency, and robustness. It combines a 12.5~Hz low-frame-rate speech tokenizer for reduced inference latency with a five-stage progressive training paradigm for coordinated language model (LM) and flow-matching model (FM) optimization. The model provides production-level control through free-style natural-language instructions and fine-grained inline tags, while supporting 16 languages, 20 Chinese dialect regions, one-pass long-form synthesis up to 3 minutes, and robust generation from noisy, reverberant, or unclear reference speech. Across SEED-TTS-Eval, CV3-Eval, instruction-following, long-form, and acoustic-robustness evaluations, Qwen-Audio-3.0-TTS achieves state-of-the-art performance on many reported dimensions or the strongest aggregate results. It also ranks first on the independent Artificial Analysis Text-to-Speech Leaderboard. These results establish Qwen-Audio-3.0-TTS as a strong foundation for production-level speech synthesis.

Citations

• IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech
• MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encoder
• Flow-GRPO: Training Flow Matching Models via Online RL
• F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization
• Qwen2.5-Omni Technical Report
• Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
• DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation
• MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
• Long-Form Speech Generation with Spoken Language Models
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
• F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching
• CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens
• E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS
• Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
• Finite Scalar Quantization: VQ-VAE Made Simple
• Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale
• An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification
• FunASR: A Fundamental End-to-End Speech Recognition Toolkit
• Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
• Robust Speech Recognition via Large-Scale Weak Supervision
• Classifier-Free Diffusion Guidance
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification
• DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Common Voice: A Massively-Multilingual Speech Corpus
• Categorical Reparameterization with Gumbel-Softmax

Cited by

• Qwen-Audio-3.0-Gen-Preview Technical Report

Related

• CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training
• GLM-TTS Technical Report
• InstructAudio: Unified speech and music generation with natural language instruction
• Review of end-to-end speech synthesis technology based on deep learning
• AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines