2019/10/25 by Yuanfeng Song, Song, Yuanfeng, Di Jiang +11 · 8 citations
Computer Science · Engineering · #Artificial intelligence #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #Computer science #Embedding #FOS: Computer and information sciences #FOS: Electrical engineering #Language model #Mechanism (biology) #Natural Language Processing Techniques #Natural language processing #Perplexity #Range (aeronautics) #Sentence #Sound (cs.SD) #Speech Recognition and Synthesis #Speech recognition #Topic Modeling #cs.CL #cs.SD #eess.AS #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1910.11496
published in arXiv (Cornell University) (Cornell University) · 5 pages, 3 figures
arxiv created 2019/10/25 · openalex publication_date 2019/10/25 · arxiv updated 2019/10/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04
Modern Automatic Speech Recognition (ASR) systems primarily rely on scores from an Acoustic Model (AM) and a Language Model (LM) to rescore the N-best lists. With the abundance of recent natural language processing advances, the information utilized by current ASR for evaluating the linguistic and semantic legitimacy of the N-best hypotheses is rather limited. In this paper, we propose a novel Learning-to-Rescore (L2RS) mechanism, which is specialized for utilizing a wide range of textual information from the state-of-the-art NLP models and automatically deciding their weights to rescore the N-best lists for ASR systems. Specifically, we incorporate features including BERT sentence embedding, topic vector, and perplexity scores produced by n-gram LM, topic modeling LM, BERT LM and RNNLM to train a rescoring model. We conduct extensive experiments based on a public dataset, and experimental results show that L2RS outperforms not only traditional rescoring methods but also its deep neural network counterparts by a substantial improvement of 20.67% in terms of NDCG@10. L2RS paves the way for developing more effective rescoring models for ASR.