2019/02/05 by Eldad Meller, Alexander M. Finkel’stein, Meller, Eldad +5 · 4 citations
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Brain Tumor Detection and Classification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.1902.01917
openalex publication_date 2019/02/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Quantization of neural networks has become common practice, driven by the need for efficient implementations of deep neural networks on embedded devices. In this paper, we exploit an oft-overlooked degree of freedom in most networks - for a given layer, individual output channels can be scaled by any factor provided that the corresponding weights of the next layer are inversely scaled. Therefore, a given network has many factorizations which change the weights of the network without changing its function. We present a conceptually simple and easy to implement method that uses this property and show that proper factorizations significantly decrease the degradation caused by quantization. We show improvement on a wide variety of networks and achieve state-of-the-art degradation results for MobileNets. While our focus is on quantization, this type of factorization is applicable to other domains such as network-pruning, neural nets regularization and network interpretability.