A Comparison of Deep Learning MOS Predictors for Speech Synthesis Quality

2022/04/05 by Alessandro Ragano, Ragano, Alessandro, Emmanouil Benetos +11
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2204.02249

openalex publication_date 2022/04/05 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

Speech synthesis quality prediction has made remarkable progress with the development of supervised and self-supervised learning (SSL) MOS predictors but some aspects related to the data are still unclear and require further study. In this paper, we evaluate several MOS predictors based on wav2vec 2.0 and the NISQA speech quality prediction model to explore the role of the training data, the influence of the system type, and the role of cross-domain features in SSL models. Our evaluation is based on the VoiceMOS challenge dataset. Results show that SSL-based models show the highest correlation and lowest mean squared error compared to supervised models. The key point of this study is that benchmarking the statistical performance of MOS predictors alone is not sufficient to rank models since potential issues hidden in the data could bias the evaluated performances.

Related

• Utilizing Self-supervised Representations for MOS Prediction
• Improving Self-Supervised Learning-based MOS Prediction Networks
• Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction
• LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement
• SAMOS: A Neural MOS Prediction Model Leveraging Semantic Representations and Acoustic Features