Fast and Low-Memory Deep Neural Networks Using Binary Matrix Factorization

2022/10/24 by Bordbar, Alireza, Kahaei, Mohammad Hossein
#FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2210.13468

Abstract

Despite the outstanding performance of deep neural networks in different applications, they are still computationally extensive and require a great number of memories. This motivates more research on reducing the resources required for implementing such networks. An efficient approach addressed for this purpose is matrix factorization, which has been shown to be effective on different networks. In this paper, we utilize binary matrix factorization and show its great efficiency in reducing the required number of resources in deep neural networks. In effect, this technique can lead to the practical implementation of such networks.

Related

• Neural Network Matrix Factorization
• Greenformer: Factorization Toolkit for Efficient Deep Neural Networks
• Fixed-point Factorized Networks
• An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks
• Refining the Structure of Neural Networks Using Matrix Conditioning