Composite Binary Decomposition Networks

2018/11/16 by You Qiaoben, Qiaoben, You, Zheng Wang +10
Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Tensor decomposition and applications #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1811.06668

arxiv created 2018/11/16 · openalex publication_date 2018/11/16 · arxiv updated 2018/11/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• ImageNet Large Scale Visual Recognition Challenge
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Neural Architecture Search with Reinforcement Learning
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Progressive Neural Architecture Search
• Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines
• Network Decoupling: From Regular to Depthwise Separable Convolutions

Related

• Matrix and tensor decompositions for training binary neural networks
• Structured Binary Neural Networks for Image Recognition
• Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
• BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
• On Study of the Binarized Deep Neural Network for Image Classification