Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers

2019/11/01 by Xishan Zhang, Shaoli Liu, Zhang, Xishan +19 · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1911.00361

We would like to withdraw the manuscript because it lacks of comparisons. The main contribution is not well verified by experiments

openalex publication_date 2019/11/01 · openalex created_date 2019/11/22 · arxiv created 2020/03/05 · arxiv updated 2020/03/06 · openalex updated_date 2026/07/28

Abstract

Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers. Recent emerged quantization technique has been applied to inference of deep neural networks for fast and efficient execution. However, directly applying quantization in training can cause significant accuracy loss, thus remaining an open challenge.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• The Pascal Visual Object Classes (VOC) Challenge
• ImageNet Large Scale Visual Recognition Challenge
• ImageNet classification with deep convolutional neural networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Fixed Point Quantization of Deep Convolutional Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Cited by

• Adaptive Precision Training for Resource Constrained Devices

Related

• Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm
• Precision Neural Network Quantization via Learnable Adaptive Modules
• Quantization Error as a Metric for Dynamic Precision Scaling in Neural Net Training
• Towards Efficient Training for Neural Network Quantization
• Attention Round for Post-Training Quantization