Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations

2020/02/17 by Yichi Zhang, Ritchie Zhao, Zhang, Yichi +9
Computer Science · Engineering · #Advanced Neural Network Applications #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.2002.07136

openalex created_date 2019/12/26 · openalex publication_date 2020/02/17 · openalex updated_date 2026/07/28

Abstract

We propose precision gating (PG), an end-to-end trainable dynamic dual-precision quantization technique for deep neural networks. PG computes most features in a low precision and only a small proportion of important features in a higher precision to preserve accuracy. The proposed approach is applicable to a variety of DNN architectures and significantly reduces the computational cost of DNN execution with almost no accuracy loss. Our experiments indicate that PG achieves excellent results on CNNs, including statically compressed mobile-friendly networks such as ShuffleNet. Compared to the state-of-the-art prediction-based quantization schemes, PG achieves the same or higher accuracy with 2.4× less compute on ImageNet. PG furthermore applies to RNNs. Compared to 8-bit uniform quantization, PG obtains a 1.2% improvement in perplexity per word with 2.7× computational cost reduction on LSTM on the Penn Tree Bank dataset. Code is available at: https://github.com/cornell-zhang/dnn-gating

Citations

• Long Short-Term Memory
• ImageNet classification with deep convolutional neural networks
• Fixed Point Quantization of Deep Convolutional Networks
• Ternary Weight Networks
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Learning Multiple Layers of Features from Tiny Images

Cited by

• OverQ: Opportunistic Outlier Quantization for Neural Network Accelerators

Related

• Mixed Precision DNNs: All you need is a good parametrization
• Stochastic Layer-Wise Precision in Deep Neural Networks
• Quantization Error as a Metric for Dynamic Precision Scaling in Neural Net Training
• BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization
• Mixed-Precision Neural Networks: A Survey