2020/11/23 by Chao-Han Huck Yang, Linda Liu, Yang, Chao-Han Huck +11
Computer Science · Engineering · #Acoustic model #Artificial Intelligence (cs.AI) #Artificial intelligence #Audio and Speech Processing (eess.AS) #Baseline (sea) #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #FOS: Electrical engineering #Language model #Machine Learning (cs.LG) #Natural Language Processing Techniques #Natural language processing #Neural and Evolutionary Computing (cs.NE) #Set (abstract data type) #Sound (cs.SD) #Speech Recognition and Synthesis #Speech processing #Speech recognition #Task (project management) #Test set #Topic Modeling #Word (group theory) #Word error rate #cs.AI #cs.CL #cs.LG #cs.NE #cs.SD #eess.AS #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2011.11715
published in arXiv (Cornell University) (Cornell University) · Accepted to IEEE Automatic Speech Recognition and Understanding (ASRU) 2021
openalex publication_date 2020/11/23 · arxiv created 2021/09/11 · arxiv updated 2021/09/14 · openalex created_date 2022/07/25 · openalex updated_date 2026/08/04
End-to-end automatic speech recognition (ASR) systems are increasingly\npopular due to their relative architectural simplicity and competitive\nperformance. However, even though the average accuracy of these systems may be\nhigh, the performance on rare content words often lags behind hybrid ASR\nsystems. To address this problem, second-pass rescoring is often applied\nleveraging upon language modeling. In this paper, we propose a second-pass\nsystem with multi-task learning, utilizing semantic targets (such as intent and\nslot prediction) to improve speech recognition performance. We show that our\nrescoring model trained with these additional tasks outperforms the baseline\nrescoring model, trained with only the language modeling task, by 1.4% on a\ngeneral test and by 2.6% on a rare word test set in terms of word-error-rate\nrelative (WERR). Our best ASR system with multi-task LM shows 4.6% WERR\ndeduction compared with RNN Transducer only ASR baseline for rare words\nrecognition.\n