Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training

2019/05/02 by Bowen Shi, Shi, Bowen, Ming Sun +10
Computer Science · Engineering · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #cs.CL #cs.SD #eess.AS #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1905.00855

NeuralPS 2018 CDNNRIA workshop

arxiv created 2019/05/02 · openalex publication_date 2019/05/02 · arxiv updated 2019/05/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we present a compression approach based on the combination of low-rank matrix factorization and quantization training, to reduce complexity for neural network based acoustic event detection (AED) models. Our experimental results show this combined compression approach is very effective. For a three-layer long short-term memory (LSTM) based AED model, the original model size can be reduced to 1% with negligible loss of accuracy. Our approach enables the feasibility of deploying AED for resource-constraint applications.

Citations

• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• On the Compression of Recurrent Neural Networks with an Application to\n LVCSR acoustic modeling for Embedded Speech Recognition

Related

• Compression of Acoustic Event Detection Models With Quantized Distillation
• Semi-supervised Acoustic Event Detection based on tri-training
• A Lottery Ticket Hypothesis Framework for Low-Complexity Device-Robust Neural Acoustic Scene Classification
• Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection
• On the efficient representation and execution of deep acoustic models