Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

2020/11/10 by Erica Cooper, Cooper, Erica, Xin Wang +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.2011.04839

openalex publication_date 2020/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We explore pretraining strategies including choice of base corpus with the aim of choosing the best strategy for zero-shot multi-speaker end-to-end synthesis. We also examine choice of neural vocoder for waveform synthesis, as well as acoustic configurations used for mel spectrograms and final audio output. We find that fine-tuning a multi-speaker model from found audiobook data that has passed a simple quality threshold can improve naturalness and similarity to unseen target speakers of synthetic speech. Additionally, we find that listeners can discern between a 16kHz and 24kHz sampling rate, and that WaveRNN produces output waveforms of a comparable quality to WaveNet, with a faster inference time.

Citations

• WaveNet: A Generative Model for Raw Audio
• Efficient Neural Audio Synthesis

Related

• Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains
• Speaker Conditional WaveRNN: Towards Universal Neural Vocoder for Unseen\n Speaker and Recording Conditions
• Speaker-independent neural formant synthesis
• Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora
• Effect of data reduction on sequence-to-sequence neural TTS