Binary Input Layer: Training of CNN models with binary input data

2018/12/09 by Robert Dürichen, Dürichen, Robert, Thomas Rocznik +5 · 1 citation
Computer Science · Engineering · #Advanced Neural Network Applications #Advanced Memory and Neural Computing #Ferroelectric and Negative Capacitance Devices

paper · pdf · doi:10.48550/arxiv.1812.03410

Abstract

For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1:92 percentage points (pp) while consuming only 2 % of the chip area.

Citations

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Cited by

• End-to-end fully-binarized network design: from Generic Learned Thermometer to Block Pruning

Related

• Input Layer Binarization with Bit-Plane Encoding
• Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance
• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
• Training Competitive Binary Neural Networks from Scratch
• Training Binary Neural Networks with Real-to-Binary Convolutions