Speaker Generation

2021/11/07 by Daisy Stanton, Matt Shannon, Stanton, Daisy +11 · 5 citations
Computer Science · Engineering · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #G.3 #I.2.7 #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and dialogue systems #cs.CL #cs.LG #cs.SD #eess.AS #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2111.05095

12 pages, 3 figures, 4 tables, appendix with 2 tables

arxiv created 2021/11/07 · openalex publication_date 2021/11/07 · arxiv updated 2021/11/10 · openalex created_date 2022/10/15 · openalex updated_date 2026/07/28

Abstract

This work explores the task of synthesizing speech in nonexistent human-sounding voices. We call this task "speaker generation", and present TacoSpawn, a system that performs competitively at this task. TacoSpawn is a recurrent attention-based text-to-speech model that learns a distribution over a speaker embedding space, which enables sampling of novel and diverse speakers. Our method is easy to implement, and does not require transfer learning from speaker ID systems. We present objective and subjective metrics for evaluating performance on this task, and demonstrate that our proposed objective metrics correlate with human perception of speaker similarity. Audio samples are available on our demo page.

Cited by

• Generating Novel and Realistic Speakers for Voice Conversion
• SequenceLayers: Sequence Processing and Streaming Neural Networks Made Easy
• Active Learning for Text-to-Speech Synthesis with Informative Sample Collection
• Eigenvoice Synthesis based on Model Editing for Speaker Generation
• TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data
• EG-VAE: A Unified Framework for Electric Guitar Tone Transfer and Removal

Related

• Mid-attribute speaker generation using optimal-transport-based interpolation of Gaussian mixture models
• Generating Speakers by Prompting Listener Impressions for Pre-trained Multi-Speaker Text-to-Speech Systems
• GC-TTS: Few-shot Speaker Adaptation with Geometric Constraints
• Low-Resource Text-to-Speech Synthesis Using Noise-Augmented Training of ForwardTacotron
• RAVE for Speech: Efficient Voice Conversion at High Sampling Rates