External Language Model Integration for Factorized Neural Transducers

2023/05/26 by Michael Levit, Levit, Michael, Sarangarajan Parthasarathy +7
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2305.17304

Abstract

We propose an adaptation method for factorized neural transducers (FNT) with external language models. We demonstrate that both neural and n-gram external LMs add significantly more value when linearly interpolated with predictor output compared to shallow fusion, thus confirming that FNT forces the predictor to act like regular language models. Further, we propose a method to integrate class-based n-gram language models into FNT framework resulting in accuracy gains similar to a hybrid setup. We show average gains of 18% WERR with lexical adaptation across various scenarios and additive gains of up to 60% WERR in one entity-rich scenario through a combination of class-based n-gram and neural LMs.

Related

• Factorized Neural Transducer for Efficient Language Model Adaptation
• Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition models
• Improved Factorized Neural Transducer Model For text-only Domain Adaptation
• Effective internal language model training and fusion for factorized transducer model
• Neural-FST Class Language Model for End-to-End Speech Recognition