2019/08/30 by Amey Agrawal, Agrawal, Amey, Rohit Karlupia +1
Computer Science · Engineering · Mathematics · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE) #cs.CV #cs.LG #cs.NE #stat.ML
paper · pdf · doi:10.48550/arxiv.1909.00052
openalex publication_date 2019/08/30 · arxiv created 2020/05/03 · arxiv updated 2020/05/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recently, in the paper "Weight Agnostic Neural Networks" Gaier & Ha utilized architecture search to find networks where the topology completely encodes the knowledge. However, architecture search in topology space is expensive. We use the existing framework of binarized networks to find performant topologies by constraining the weights to be either, zero or one. We show that such topologies achieve performance similar to standard networks while pruning more than 99% weights. We further demonstrate that these topologies can perform tasks using constant weights without any explicit tuning. Finally, we discover that in our setup each neuron acts like a NOR gate, virtually learning a digital circuit. We demonstrate the efficacy of our approach on computer vision datasets.