Fixed-point Factorized Networks

2016/11/07 by Peisong Wang, Jian Cheng, Wang, Peisong +1
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1611.01972

openalex publication_date 2016/11/07 · arxiv created 2017/08/29 · arxiv updated 2017/08/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years, Deep Neural Networks (DNN) based methods have achieved remarkable performance in a wide range of tasks and have been among the most powerful and widely used techniques in computer vision. However, DNN-based methods are both computational-intensive and resource-consuming, which hinders the application of these methods on embedded systems like smart phones. To alleviate this problem, we introduce a novel Fixed-point Factorized Networks (FFN) for pretrained models to reduce the computational complexity as well as the storage requirement of networks. The resulting networks have only weights of -1, 0 and 1, which significantly eliminates the most resource-consuming multiply-accumulate operations (MACs). Extensive experiments on large-scale ImageNet classification task show the proposed FFN only requires one-thousandth of multiply operations with comparable accuracy.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FitNets: Hints for Thin Deep Nets
• Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation
• Distilling the Knowledge in a Neural Network
• Deep Learning with Limited Numerical Precision
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• ImageNet Large Scale Visual Recognition Challenge
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• 8-Bit Approximations for Parallelism in Deep Learning
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Predicting Parameters in Deep Learning
• Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications

Related

• A Survey of Methods for Low-Power Deep Learning and Computer Vision
• Fast and Low-Memory Deep Neural Networks Using Binary Matrix Factorization
• Factorial Convolution Neural Networks
• Task-Driven Fixation Network: An Efficient Architecture with Fixation Selection
• An Analysis of Deep Neural Network Models for Practical Applications