Diff-TTS: A Denoising Diffusion Model for Text-to-Speech

2021/04/03 by Myeonghun Jeong, Hyeongju Kim, Jeong, Myeonghun +7 · 8 citations
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2104.01409

openalex publication_date 2021/04/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Although neural text-to-speech (TTS) models have attracted a lot of attention and succeeded in generating human-like speech, there is still room for improvements to its naturalness and architectural efficiency. In this work, we propose a novel non-autoregressive TTS model, namely Diff-TTS, which achieves highly natural and efficient speech synthesis. Given the text, Diff-TTS exploits a denoising diffusion framework to transform the noise signal into a mel-spectrogram via diffusion time steps. In order to learn the mel-spectrogram distribution conditioned on the text, we present a likelihood-based optimization method for TTS. Furthermore, to boost up the inference speed, we leverage the accelerated sampling method that allows Diff-TTS to generate raw waveforms much faster without significantly degrading perceptual quality. Through experiments, we verified that Diff-TTS generates 28 times faster than the real-time with a single NVIDIA 2080Ti GPU.

Citations

• Tacotron: Towards End-to-End Speech Synthesis
• Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
• Denoising Diffusion Probabilistic Models
• Denoising Diffusion Implicit Models

Cited by

• Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models
• Position: Towards Responsible Evaluation for Text-to-Speech
• An Octave-based Multi-Resolution CQT Architecture for Diffusion-based Audio Generation
• Length-Aware Rotary Position Embedding for Text-Speech Alignment
• DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibration
• A Survey on Neural Speech Synthesis
• Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models
• Enhancing In-the-Wild Speech Emotion Conversion with Resynthesis-based Duration Modeling

Related

• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
• ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
• E3 TTS: Easy End-to-End Diffusion-based Text to Speech
• CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models
• FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis