MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network

2021/02/27 by Yichong Leng, Leng, Yichong, Xu Tan +11 · 16 citations
Computer Science · Engineering · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Natural Language Processing Techniques #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #cs.SD #eess.AS #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2103.00110

Accepted by ICASSP 2021

arxiv created 2021/02/27 · openalex publication_date 2021/02/27 · arxiv updated 2021/03/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Mean opinion score (MOS) is a popular subjective metric to assess the quality of synthesized speech, and usually involves multiple human judges to evaluate each speech utterance. To reduce the labor cost in MOS test, multiple methods have been proposed to automatically predict MOS scores. To our knowledge, for a speech utterance, all previous works only used the average of multiple scores from different judges as the training target and discarded the score of each individual judge, which did not well exploit the precious MOS training data. In this paper, we propose MBNet, a MOS predictor with a mean subnet and a bias subnet to better utilize every judge score in MOS datasets, where the mean subnet is used to predict the mean score of each utterance similar to that in previous works, and the bias subnet to predict the bias score (the difference between the mean score and each individual judge score) and capture the personal preference of individual judges. Experiments show that compared with MOSNet baseline that only leverages mean score for training, MBNet improves the system-level spearmans rank correlation co-efficient (SRCC) by 2.9% on VCC 2018 dataset and 6.7% on VCC 2016 dataset.

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• NOTES ON THE HISTORY OF CORRELATION
• AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech
• Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification

Cited by

• ORCA: Open-ended Response Correctness Assessment for Audio Question Answering
• Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddings
• From Evaluation to Optimization: Neural Speech Assessment for Downstream Applications
• Towards Frame-level Quality Predictions of Synthetic Speech
• Leveraging LLMs for Scalable Non-intrusive Speech Quality Assessment
• Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition
• MambaRate: Speech Quality Assessment Across Different Sampling Rates
• JSQA: Speech Quality Assessment with Perceptually-Inspired Contrastive Pretraining Based on JND Audio Pairs
• RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio
• Learning to assess subjective impressions from speech
• Selecting N-lowest scores for training MOS prediction models
• Rethinking Mean Opinion Scores in Speech Quality Assessment: Aggregation through Quantized Distribution Fitting
• SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction
• SHEET: A Multi-purpose Open-source Speech Human Evaluation Estimation Toolkit
• Multi-objective non-intrusive hearing-aid speech assessment model
• APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech

Related

• MOSPC: MOS Prediction Based on Pairwise Comparison
• Generalization Ability of MOS Prediction Networks
• Selecting N-lowest scores for training MOS prediction models
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech
• Fusion of Self-supervised Learned Models for MOS Prediction