Expediting TTS Synthesis with Adversarial Vocoding

2019/04/16 by Neekhara, Paarth, Donahue, Chris, Puckette, Miller +2
#Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.1904.07944

Abstract

Recent approaches in text-to-speech (TTS) synthesis employ neural network strategies to vocode perceptually-informed spectrogram representations directly into listenable waveforms. Such vocoding procedures create a computational bottleneck in modern TTS pipelines. We propose an alternative approach which utilizes generative adversarial networks (GANs) to learn mappings from perceptually-informed spectrograms to simple magnitude spectrograms which can be heuristically vocoded. Through a user study, we show that our approach significantly outperforms naïve vocoding strategies while being hundreds of times faster than neural network vocoders used in state-of-the-art TTS systems. We also show that our method can be used to achieve state-of-the-art results in unsupervised synthesis of individual words of speech.

Related

• Waveform generation for text-to-speech synthesis using pitch-synchronous\n multi-scale generative adversarial networks
• Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram\n Generation with Adversarial Style Combination for Speech Synthesis
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis
• DSPGAN: a GAN-based universal vocoder for high-fidelity TTS by time-frequency domain supervision from DSP