An Effective Strategy for Modeling Score Ordinality and Non-uniform Intervals in Automated Speaking Assessment

2025/08/27 by Lo, Tien-Hong, Chen, Szu-Yu, Sung, Yao-Ting +1
#Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2509.03372

Abstract

A recent line of research on automated speaking assessment (ASA) has benefited from self-supervised learning (SSL) representations, which capture rich acoustic and linguistic patterns in non-native speech without underlying assumptions of feature curation. However, speech-based SSL models capture acoustic-related traits but overlook linguistic content, while text-based SSL models rely on ASR output and fail to encode prosodic nuances. Moreover, most prior arts treat proficiency levels as nominal classes, ignoring their ordinal structure and non-uniform intervals between proficiency labels. To address these limitations, we propose an effective ASA approach combining SSL with handcrafted indicator features via a novel modeling paradigm. We further introduce a multi-margin ordinal loss that jointly models both the score ordinality and non-uniform intervals of proficiency labels. Extensive experiments on the TEEMI corpus show that our method consistently outperforms strong baselines and generalizes well to unseen prompts.

Citations

• CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss
• The Llama 3 Herd of Models
• An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution
• A Hierarchical Context-aware Modeling Approach for Multi-aspect and Multi-granular Pronunciation Assessment
• Proficiency assessment of L2 spoken English using wav2vec 2.0
• CEFR-Based Sentence Difficulty Annotation and Assessment
• 3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
• Attention Is All You Need
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution
• Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency
• Optimizing Automatic Speech Assessment: W-RankSim Regularization and Hybrid Feature Fusion Strategies
• Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring
• A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions