CondenseNet: An Efficient DenseNet using Learned Group Convolutions

2017/11/25 by Gao Huang, Shichen Liu, Huang, Gao +5 · 19 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.1711.09224

Abstract

Deep neural networks are increasingly used on mobile devices, where computational resources are limited. In this paper we develop CondenseNet, a novel network architecture with unprecedented efficiency. It combines dense connectivity with a novel module called learned group convolution. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard group convolutions, allowing for efficient computation in practice. Our experiments show that CondenseNets are far more efficient than state-of-the-art compact convolutional networks such as MobileNets and ShuffleNets.

Citations

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Spatially Adaptive Computation Time for Residual Networks

Cited by

• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• ELASTIC: Improving CNNs with Dynamic Scaling Policies
• Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
• Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
• Emulating Human-like Adaptive Vision for Efficient and Flexible Machine Visual Perception
• DeepTraverse: A Depth-First Search Inspired Network for Algorithmic Visual Understanding
• Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
• Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
• Reverse Convolution and Its Applications to Image Restoration
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
• TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• Accelerated CNN Training Through Gradient Approximation
• Fast AI Model Splitting over Edge Networks
• Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
• Path-Level Network Transformation for Efficient Architecture Search
• Deep Learning Based Text Classification: A Comprehensive Review
• Full-stack Optimization for Accelerating CNNs with FPGA Validation

Related

• HGC: Hierarchical Group Convolution for Highly Efficient Neural Network
• Merging and Evolution: Improving Convolutional Neural Networks for Mobile Applications
• ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• HENet:A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage