Accelerating recurrent neural network language model based online speech\n recognition system

2018/01/30 by Kyungmin Lee, Chiyoun Park, Lee, Kyungmin +5
Computer Science · #Speech Recognition and Synthesis #Natural Language Processing Techniques #Network Packet Processing and Optimization

paper · pdf · doi:10.48550/arxiv.1801.09866

Abstract

This paper presents methods to accelerate recurrent neural network based\nlanguage models (RNNLMs) for online speech recognition systems. Firstly, a\nlossy compression of the past hidden layer outputs (history vector) with\ncaching is introduced in order to reduce the number of LM queries. Next, RNNLM\ncomputations are deployed in a CPU-GPU hybrid manner, which computes each layer\nof the model on a more advantageous platform. The added overhead by data\nexchanges between CPU and GPU is compensated through a frame-wise batching\nstrategy. The performance of the proposed methods evaluated on LibriSpeech test\nsets indicates that the reduction in history vector precision improves the\naverage recognition speed by 1.23 times with minimum degradation in accuracy.\nOn the other hand, the CPU-GPU hybrid parallelization enables RNNLM based\nreal-time recognition with a four times improvement in speed.\n

Related

• Applying GPGPU to Recurrent Neural Network Language Model based Fast\n Network Search in the Real-Time LVCSR
• Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
• FPGA-Based Low-Power Speech Recognition with Recurrent Neural Networks
• A review of on-device fully neural end-to-end automatic speech recognition algorithms
• Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer