2017/07/01 by Yoonho Boo, Boo, Yoonho, Wonyong Sung +1
Computer Science · Engineering · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Sparse and Compressive Sensing Techniques
paper · pdf · doi:10.48550/arxiv.1707.03684
openalex publication_date 2017/07/01 · openalex created_date 2022/10/03 · openalex updated_date 2026/07/28
Deep neural networks (DNNs) usually demand a large amount of operations for\nreal-time inference. Especially, fully-connected layers contain a large number\nof weights, thus they usually need many off-chip memory accesses for inference.\nWe propose a weight compression method for deep neural networks, which allows\nvalues of +1 or -1 only at predetermined positions of the weights so that\ndecoding using a table can be conducted easily. For example, the structured\nsparse (8,2) coding allows at most two non-zero values among eight weights.\nThis method not only enables multiplication-free DNN implementations but also\ncompresses the weight storage by up to x32 compared to floating-point networks.\nWeight distribution normalization and gradual pruning techniques are applied to\nmitigate the performance degradation. The experiments are conducted with\nfully-connected deep neural networks and convolutional neural networks.\n