Speaker-independent neural formant synthesis

2023/06/02 by Pablo Pérez Zarazaga, Zofia Malisz, Zarazaga, Pablo Pérez +5
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2306.01957

openalex publication_date 2023/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We describe speaker-independent speech synthesis driven by a small set of phonetically meaningful speech parameters such as formant frequencies. The intention is to leverage deep-learning advances to provide a highly realistic signal generator that includes control affordances required for stimulus creation in the speech sciences. Our approach turns input speech parameters into predicted mel-spectrograms, which are rendered into waveforms by a pre-trained neural vocoder. Experiments with WaveNet and HiFi-GAN confirm that the method achieves our goals of accurate control over speech parameters combined with high perceptual audio quality. We also find that the small set of phonetically relevant speech parameters we use is sufficient to allow for speaker-independent synthesis (a.k.a. universal vocoding).

Related

• HiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters
• Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations
• RawNet: Fast End-to-End Neural Vocoder
• Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains
• VNet: A GAN-based Multi-Tier Discriminator Network for Speech Synthesis Vocoders