Text-to-speech synthesis from dark data with evaluation-in-the-loop data selection

2022/10/26 by Kentaro Seki, Seki, Kentaro, Shinnosuke Takamichi +5 · 4 citations
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2210.14850

openalex publication_date 2022/10/26 · openalex created_date 2022/11/02 · openalex updated_date 2026/07/28

Abstract

This paper proposes a method for selecting training data for text-to-speech (TTS) synthesis from dark data. TTS models are typically trained on high-quality speech corpora that cost much time and money for data collection, which makes it very challenging to increase speaker variation. In contrast, there is a large amount of data whose availability is unknown (a.k.a, "dark data"), such as YouTube videos. To utilize data other than TTS corpora, previous studies have selected speech data from the corpora on the basis of acoustic quality. However, considering that TTS models robust to data noise have been proposed, we should select data on the basis of its importance as training data to the given TTS model, not the quality of speech itself. Our method with a loop of training and evaluation selects training data on the basis of the automatically predicted quality of synthetic speech of a given TTS model. Results of evaluations using YouTube data reveal that our method outperforms the conventional acoustic-quality-based method.

Cited by

• Evaluation of preprocessing pipelines in the creation of in-the-wild TTS datasets
• AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation
• Active Learning for Text-to-Speech Synthesis with Informative Sample Collection
• TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data

Related

• TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data
• Training Text-to-Speech Model with Purely Synthetic Data: Feasibility, Sensitivity, and Generalization Capability
• Text-To-Speech Synthesis In The Wild
• TTS-by-TTS: TTS-driven Data Augmentation for Fast and High-Quality Speech Synthesis
• Comparative Analysis of Transfer Learning in Deep Learning Text-to-Speech Models on a Few-Shot, Low-Resource, Customized Dataset