Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification

2020/07/16 by Yeunju Choi, Choi, Yeunju, Youngmoon Jung +3 · 1 citation
Computer Science · Medicine · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #Voice and Speech Disorders #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2007.08267

openalex publication_date 2020/07/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Several studies have proposed deep-learning-based models to predict the mean opinion score (MOS) of synthesized speech, showing the possibility of replacing human raters. However, inter- and intra-rater variability in MOSs makes it hard to ensure the high performance of the models. In this paper, we propose a multi-task learning (MTL) method to improve the performance of a MOS prediction model using the following two auxiliary tasks: spoofing detection (SD) and spoofing type classification (STC). Besides, we use the focal loss to maximize the synergy between SD and STC for MOS prediction. Experiments using the MOS evaluation results of the Voice Conversion Challenge 2018 show that proposed MTL with two auxiliary tasks improves MOS prediction. Our proposed model achieves up to 11.6% relative improvement in performance over the baseline model.

Citations

• NOTES ON THE HISTORY OF CORRELATION
• The Proof and Measurement of Association between Two Things
• AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech
• An Overview of Multi-Task Learning in Deep Neural Networks
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Cited by

• MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network

Related

• Improving Self-Supervised Learning-based MOS Prediction Networks
• Fusion of Self-supervised Learned Models for MOS Prediction
• DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores
• SAMOS: A Neural MOS Prediction Model Leveraging Semantic Representations and Acoustic Features
• Generalization Ability of MOS Prediction Networks