Rotation Invariant Quantization for Model Compression

2023/03/03 by Kampeas, Joseph, Nahshan, Yury, Kremer, Hanoch +4
#Artificial Intelligence (cs.AI) #E.4 #FOS: Computer and information sciences #I.2.4 #Information Theory (cs.IT) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2303.03106

Abstract

Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. In this study, we investigate the rate-distortion tradeoff for NN model compression. First, we suggest a Rotation-Invariant Quantization (RIQ) technique that utilizes a single parameter to quantize the entire NN model, yielding a different rate at each layer, i.e., mixed-precision quantization. Then, we prove that our rotation-invariant approach is optimal in terms of compression. We rigorously evaluate RIQ and demonstrate its capabilities on various models and tasks. For example, RIQ facilitates × 19.4 and × 52.9 compression ratios on pre-trained VGG dense and pruned models, respectively, with <0.4% accuracy degradation. Code is available in \hrefhttps://github.com/ehaleva/RIQgithub.com/ehaleva/RIQ.

Related

• Rate-Distortion Optimized Post-Training Quantization for Learned Image Compression
• Rate Distortion For Model Compression: From Theory To Practice
• Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning
• Retraining-Based Iterative Weight Quantization for Deep Neural Networks
• Sensitivity-Aware Post-Training Quantization for Deep Neural Networks