Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling

2020/10/08 by Jonathan Shen, Jia Ye, Ye Jia +12 · 3 citations
Computer Science · Engineering · #Advanced Chemical Sensor Technologies #Blind Source Separation Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Neural Networks and Applications #Sound (cs.SD) #cs.CL #cs.SD

paper · pdf · doi:10.48550/arxiv.2010.04301

openalex publication_date 2020/10/08 · openalex created_date 2020/10/15 · arxiv created 2021/05/11 · arxiv updated 2021/05/12 · openalex updated_date 2026/07/28

Abstract

This paper presents Non-Attentive Tacotron based on the Tacotron 2 text-to-speech model, replacing the attention mechanism with an explicit duration predictor. This improves robustness significantly as measured by unaligned duration ratio and word deletion rate, two metrics introduced in this paper for large-scale robustness evaluation using a pre-trained speech recognition model. With the use of Gaussian upsampling, Non-Attentive Tacotron achieves a 5-scale mean opinion score for naturalness of 4.41, slightly outperforming Tacotron 2. The duration predictor enables both utterance-wide and per-phoneme control of duration at inference time. When accurate target durations are scarce or unavailable in the training data, we propose a method using a fine-grained variational auto-encoder to train the duration predictor in a semi-supervised or unsupervised manner, with results almost as good as supervised training.

Citations

• Attention-Based Models for Speech Recognition
• Auto-Encoding Variational Bayes
• A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
• Non-Autoregressive Neural Machine Translation
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• DurIAN: Duration Informed Attention Network For Multimodal Synthesis
• AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN
• TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model
• End-to-End Adversarial Text-to-Speech
• Textual Echo Cancellation
• Parallel Tacotron: Non-Autoregressive and Controllable TTS

Cited by

• SpeakStream: Streaming Text-to-Speech with Interleaved Data
• VRAIN-UPV MLLP's system for the Blizzard Challenge 2021
• Voice Cloning: Comprehensive Survey

Related

• Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling
• Parallel Tacotron: Non-Autoregressive and Controllable TTS
• Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech
• Teacher-Student Training for Robust Tacotron-based TTS
• FeatherTTS: Robust and Efficient attention based Neural TTS