4-bit Quantization of LSTM-based Speech Recognition Models

2021/08/27 by Andrea Fasoli, Fasoli, Andrea, Chia‐Yu Chen +20 · 2 citations
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #I.2.6 #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2108.12074

openalex publication_date 2021/08/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We investigate the impact of aggressive low-precision representations of weights and activations in two families of large LSTM-based architectures for Automatic Speech Recognition (ASR): hybrid Deep Bidirectional LSTM - Hidden Markov Models (DBLSTM-HMMs) and Recurrent Neural Network - Transducers (RNN-Ts). Using a 4-bit integer representation, a naïve quantization approach applied to the LSTM portion of these models results in significant Word Error Rate (WER) degradation. On the other hand, we show that minimal accuracy loss is achievable with an appropriate choice of quantizers and initializations. In particular, we customize quantization schemes depending on the local properties of the network, improving recognition performance while limiting computational time. We demonstrate our solution on the Switchboard (SWB) and CallHome (CH) test sets of the NIST Hub5-2000 evaluation. DBLSTM-HMMs trained with 300 or 2000 hours of SWB data achieves

Citations

• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Optimizing Speech Recognition For The Edge
• Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation

Cited by

• Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision
• Context-Driven Dynamic Pruning for Large Speech Foundation Models

Related

• 4-bit Conformer with Native Quantization Aware Training for Speech Recognition
• On the efficient representation and execution of deep acoustic models
• Efficient Speech Representation Learning with Low-Bit Quantization
• Towards Green ASR: Lossless 4-bit Quantization of a Hybrid TDNN System on the 300-hr Switchboard Corpus
• Quantization of Acoustic Model Parameters in Automatic Speech Recognition Framework