Binarized Weight Error Networks With a Transition Regularization Term

2021/05/09 by Savas Ozkan, Savaş Özkan, Gözde Bozdağı Akar +3
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Image Enhancement Techniques #cs.CV

paper · pdf · doi:10.48550/arxiv.2105.03897

Submitted to ICIP 2021

arxiv created 2021/05/09 · openalex publication_date 2021/05/09 · arxiv updated 2021/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet classification with deep convolutional neural networks
• Ternary Weight Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Trained Ternary Quantization
• LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

Related

• Regularized Binary Network Training
• OvSW: Overcoming Silent Weights for Accurate Binary Neural Networks
• Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket
• Bimodal Distributed Binarized Neural Networks
• Training Compact Neural Networks with Binary Weights and Low Precision Activations