2024/09/18 by Geng, Haopeng, Daisuke Saito, Saito, Daisuke +2 · 1 citation
Computer Science · Psychology · #Audio and Speech Processing (eess.AS) #Computer science #FOS: Computer and information sciences #FOS: Electrical engineering #Indirect speech #Linguistics #Psychology #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #Speech recognition #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2409.11742
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/09/18 · openalex created_date 2024/10/25 · openalex updated_date 2026/07/28
Evaluating speech intelligibility is a critical task in computer-aided language learning systems. Traditional methods often rely on word error rates (WER) provided by automatic speech recognition (ASR) as intelligibility scores. However, this approach has significant limitations due to notable differences between human speech recognition (HSR) and ASR. A promising alternative is to involve a native (L1) speaker in shadowing what nonnative (L2) speakers say. Breakdowns or mispronunciations in the L1 speaker's shadowing utterance can serve as indicators for assessing L2 speech intelligibility. In this study, we propose a speech generation system that simulates the L1 shadowing process using voice conversion (VC) techniques and latent speech representations. Our experimental results demonstrate that this method effectively replicates the L1 shadowing process, offering an innovative tool to evaluate L2 speech intelligibility. Notably, systems that utilize self-supervised speech representations (S3R) show a higher degree of similarity to real L1 shadowing utterances in both linguistic accuracy and naturalness.