SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size

2016/02/24 by Forrest N. Iandola, Iandola, Forrest N., Song Han +9 · 2 voices · 57 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.AI #cs.CV

paper · pdf · doi:10.48550/arxiv.1602.07360

arxiv published 2016/02/24 · arxiv updated 2016/11/04

Abstract

Recent research on deep neural networks has focused primarily on improving accuracy. For a given accuracy level, it is typically possible to identify multiple DNN architectures that achieve that accuracy level. With equivalent accuracy, smaller DNN architectures offer at least three advantages: (1) Smaller DNNs require less communication across servers during distributed training. (2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car. (3) Smaller DNNs are more feasible to deploy on FPGAs and other hardware with limited memory. To provide all of these advantages, we propose a small DNN architecture called SqueezeNet. SqueezeNet achieves AlexNet-level accuracy on ImageNet with 50x fewer parameters. Additionally, with model compression techniques we are able to compress SqueezeNet to less than 0.5MB (510x smaller than AlexNet). The SqueezeNet architecture is available for download here: https://github.com/DeepScale/SqueezeNet

Cited by

• Semi-Supervised Online Learning on the Edge by Transforming Knowledge from Teacher Models
• A Comparative Analysis of Semiconductor Wafer Map Defect Detection with Image Transformer
• GlimmerNet: A Lightweight Grouped Dilated Depthwise Convolutions for UAV-Based Emergency Monitoring
• Zero-Shot Textual Explanations via Translating Decision-Critical Features
• Learning Single-Image Super-Resolution in the JPEG Compressed Domain
• Tackling Tuberculosis: A Comparative Dive into Machine Learning for Tuberculosis Detection
• IGen: Scalable Data Generation for Robot Learning from Open-World Images
• Using MLIR Transform to Design Sliced Convolution Algorithm
• Teacher-Guided One-Shot Pruning via Context-Aware Knowledge Distillation
• A lightweight progressive aggregation network for multi-contrast MRI super-resolution
• CompressNAS : A Fast and Efficient Technique for Model Compression using Decomposition
• Comparative Study of CNN Architectures for Binary Classification of Horses and Motorcycles in the VOC 2008 Dataset
• On the Optimization of Model Aggregation for Federated Learning at the Network Edge
• Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Models
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Leveraging Large-Scale Face Datasets for Deep Periocular Recognition via Ocular Cropping
• Exploring Complementarity and Explainability in CNNs for Periocular Verification Across Acquisition Distances
• Send Less, Save More: Energy-Efficiency Benchmark of Embedded CNN Inference vs. Data Transmission in IoT
• Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples
• Lightweight CycleGAN Models for Cross-Modality Image Transformation and Experimental Quality Assessment in Fluorescence Microscopy
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition
• Learning Task-Agnostic Representations through Multi-Teacher Distillation
• Removing Cost Volumes from Optical Flow Estimators
• Real-Time Crowd Counting for Embedded Systems with Lightweight Architecture
• CauchyNet: Compact and Data-Efficient Learning using Holomorphic Activation Functions
• KTBox: A Modular LaTeX Framework for Semantic Color, Structured Highlighting, and Scholarly Communication
• Computing frustration and near-monotonicity in deep neural networks
• AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs
• Graph Your Own Prompt
• Targeted perturbations reveal brain-like local coding axes in robustified, but not standard, ANN-based brain models
• Spatiotemporal Radar Gesture Recognition with Hybrid Spiking Neural Networks: Balancing Accuracy and Efficiency
• MonoCon: A general framework for learning ultra-compact high-fidelity representations using monotonicity constraints
• MS-YOLO: Infrared Object Detection for Edge Deployment via MobileNetV4 and SlideLoss
• Homodyne Photonic Tensor Processor exceeds 1,000-TOPS
• DL-QC-fNIRS: a deep learning tool for automated quality control in functional near-infrared spectroscopy signals
• Deep Lookup Network
• PerceptronCARE: A Deep Learning-Based Intelligent Teleophthalmology Application for Diabetic Retinopathy Diagnosis
• SugarcaneShuffleNet: A Very Fast, Lightweight Convolutional Neural Network for Diagnosis of 15 Sugarcane Leaf Diseases
• Point-Plane Projections for Accurate LiDAR Semantic Segmentation in Small Data Scenarios
• NAT: Learning to Attack Neurons for Enhanced Adversarial Transferability
• NeuroDeX: Unlocking Diverse Support in Decompiling Deep Neural Network Executables
• Leveraging Transfer Learning and Mobile-enabled Convolutional Neural Networks for Improved Arabic Handwritten Character Recognition
• NM-Hebb: Coupling Local Hebbian Plasticity with Metric Learning for More Accurate and Interpretable CNNs
• TAIGen: Training-Free Adversarial Image Generation via Diffusion Models
• Towards SISO Bistatic Sensing for ISAC
• 3D latent diffusion models for parameterizing and history matching multiscenario facies systems
• MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI
• Synaptic Pruning: A Biological Inspiration for Deep Learning Regularization
• Lightweight Multi-Scale Feature Extraction with Fully Connected LMF Layer for Salient Object Detection
• ULU: A Unified Activation Function
• Energy-Efficient Real-Time 4-Stage Sleep Classification at 10-Second Resolution: A Comprehensive Study
• CADD: Context aware disease deviations via restoration of brain images using normative conditional diffusion models
• DySTop
• Calibrated Prediction Set in Fault Detection with Risk Guarantees via Significance Tests
• Analysis of Hyperparameter Optimization Effects on Lightweight Deep Models for Real-Time Image Classification
• Agentic Privacy-Preserving Machine Learning
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning

Discussions

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters [hn, 3 points, 0 comments]
• .@mwakanosya @_lab41 @dl_weekly Blogged about by us, paper from Iandola et. al: https://arxiv.org/abs/1602.07360 #deeplearning #squeezenet 😉 [bsky, 0 points, 0 comments]

Related

• SlimNets: An Exploration of Deep Model Compression and Acceleration
• DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis
• Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets