MWQ: Multiscale Wavelet Quantized Neural Networks

2021/03/09 by Qigong Sun, Yan Ren, Sun, Qigong +9 · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Image and Signal Denoising Methods #Medical Image Segmentation Techniques

paper · pdf · doi:10.48550/arxiv.2103.05363

openalex publication_date 2021/03/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Model quantization can reduce the model size and computational latency, it has become an essential technique for the deployment of deep neural networks on resourceconstrained hardware (e.g., mobile phones and embedded devices). The existing quantization methods mainly consider the numerical elements of the weights and activation values, ignoring the relationship between elements. The decline of representation ability and information loss usually lead to the performance degradation. Inspired by the characteristics of images in the frequency domain, we propose a novel multiscale wavelet quantization (MWQ) method. This method decomposes original data into multiscale frequency components by wavelet transform, and then quantizes the components of different scales, respectively. It exploits the multiscale frequency and spatial information to alleviate the information loss caused by quantization in the spatial domain. Because of the flexibility of MWQ, we demonstrate three applications (e.g., model compression, quantized network optimization, and information enhancement) on the ImageNet and COCO datasets. Experimental results show that our method has stronger representation ability and can play an effective role in quantized neural networks.

Citations

• A theory for multiresolution signal decomposition: the wavelet representation
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Ternary Weight Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Trained Ternary Quantization
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• AdaBits: Neural Network Quantization with Adaptive Bit-Widths

Cited by

• Quantized Neural Networks via -1, +1 Encoding Decomposition and Acceleration

Related

• One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustment
• Weight Normalization based Quantization for Deep Neural Network Compression
• Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1
• Quantized Convolutional Neural Networks for Mobile Devices
• Quantized Neural Networks via -1, +1 Encoding Decomposition and Acceleration