Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

2015/03/12 by Zhiyong Cheng, Daniel Soudry, Cheng, Zhiyong +5 · 3 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #cs.CV #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.1503.03562

8 pages with 1 figures and 4 tables

openalex publication_date 2015/03/12 · arxiv created 2015/03/22 · arxiv updated 2015/03/24 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

Compared to Multilayer Neural Networks with real weights, Binary Multilayer Neural Networks (BMNNs) can be implemented more efficiently on dedicated hardware. BMNNs have been demonstrated to be effective on binary classification tasks with Expectation BackPropagation (EBP) algorithm on high dimensional text datasets. In this paper, we investigate the capability of BMNNs using the EBP algorithm on multiclass image classification tasks. The performances of binary neural networks with multiple hidden layers and different numbers of hidden units are examined on MNIST. We also explore the effectiveness of image spatial filters and the dropout technique in BMNNs. Experimental results on MNIST dataset show that EBP can obtain 2.12% test error with binary weights and 1.66% test error with real weights, which is comparable to the results of standard BackPropagation algorithm on fully connected MNNs.

Citations

• Gradient-based learning applied to document recognition
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups

Cited by

• PokeBNN: A Binary Pursuit of Lightweight Accuracy
• Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers
• Fixed-point Factorized Networks

Related

• An experimental comparative study of backpropagation and alternatives for training binary neural networks for image classification
• 1 bit is all we need: binary normalized neural networks
• Bitwise Neural Networks
• BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training
• On Study of the Binarized Deep Neural Network for Image Classification