Structured Convolution Matrices for Energy-efficient Deep learning

2016/06/08 by Rathinakumar Appuswamy, Tapan K. Nayak, Appuswamy, Rathinakumar +15
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1606.02407

openalex publication_date 2016/06/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We derive a relationship between network representation in energy-efficient neuromorphic architectures and block Toplitz convolutional matrices. Inspired by this connection, we develop deep convolutional networks using a family of structured convolutional matrices and achieve state-of-the-art trade-off between energy efficiency and classification accuracy for well-known image recognition tasks. We also put forward a novel method to train binary convolutional networks by utilising an existing connection between noisy-rectified linear units and binary activations.

Citations

• Compressing Deep Convolutional Networks using Vector Quantization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Improving neural networks by preventing co-adaptation of feature detectors
• On the Convergence Properties of the EM Algorithm
• ImageNet classification with deep convolutional neural networks
• Noisy Activation Functions
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks
• Learning Multiple Layers of Features from Tiny Images

Related

• Unified Sparse-Matrix Representations for Diverse Neural Architectures
• Learnable Heterogeneous Convolution: Learning both topology and strength
• Optimal Approach for Image Recognition using Deep Convolutional Architecture
• Learning Deep Matrix Representations
• Computation-efficient Deep Learning for Computer Vision: A Survey