2021/06/24 by Raahil Shah, Kamil Pokora, Shah, Raahil +13
Computer Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Sound (cs.SD) #Speech Recognition and Synthesis #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2106.12896
openalex publication_date 2021/06/24 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Whilst recent neural text-to-speech (TTS) approaches produce high-quality\nspeech, they typically require a large amount of recordings from the target\nspeaker. In previous work, a 3-step method was proposed to generate\nhigh-quality TTS while greatly reducing the amount of data required for\ntraining. However, we have observed a ceiling effect in the level of\nnaturalness achievable for highly expressive voices when using this approach.\nIn this paper, we present a method for building highly expressive TTS voices\nwith as little as 15 minutes of speech data from the target speaker. Compared\nto the current state-of-the-art approach, our proposed improvements close the\ngap to recordings by 23.3% for naturalness of speech and by 16.3% for speaker\nsimilarity. Further, we match the naturalness and speaker similarity of a\nTacotron2-based full-data (~10 hours) model using only 15 minutes of target\nspeaker data, whereas with 30 minutes or more, we significantly outperform it.\nThe following improvements are proposed: 1) changing from an autoregressive,\nattention-based TTS model to a non-autoregressive model replacing attention\nwith an external duration model and 2) an additional Conditional Generative\nAdversarial Network (cGAN) based fine-tuning step.\n