XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks

2016/03/16 by Mohammad Rastegari, Rastegari, Mohammad, Vicente Ordonez +6 · 1 voice · 61 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #cs.CV

paper · pdf · doi:10.48550/arxiv.1603.05279

openalex publication_date 2016/03/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose two efficient approximations to standard convolutional neural\nnetworks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks,\nthe filters are approximated with binary values resulting in 32x memory saving.\nIn XNOR-Networks, both the filters and the input to convolutional layers are\nbinary. XNOR-Networks approximate convolutions using primarily binary\noperations. This results in 58x faster convolutional operations and 32x memory\nsavings. XNOR-Nets offer the possibility of running state-of-the-art networks\non CPUs (rather than GPUs) in real-time. Our binary networks are simple,\naccurate, efficient, and work on challenging visual tasks. We evaluate our\napproach on the ImageNet classification task. The classification accuracy with\na Binary-Weight-Network version of AlexNet is only 2.9% less than the\nfull-precision AlexNet (in top-1 measure). We compare our method with recent\nnetwork binarization methods, BinaryConnect and BinaryNets, and outperform\nthese methods by large margins on ImageNet, more than 16% in top-1 accuracy.\n

Citations

• Big Neural Networks Waste Capacity
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Deep Residual Learning for Image Recognition
• Predicting Parameters in Deep Learning

Cited by

• Task-Oriented Communication with Hybrid-Precision Models
• Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
• 1 bit is all we need: binary normalized neural networks
• X-pSRAM: A Photonic SRAM with Embedded XOR Logic for Ultra-Fast In-Memory Computing
• A Principled Bayesian Framework for Training Binary and Spiking Neural Networks
• Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
• Binary Neural Network Implementation for Handwritten Digit Recognition on FPGA
• Hardware Software Optimizations for Fast Model Recovery on Reconfigurable Architectures
• BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training
• INSIGHT: An Interpretable Neural Vision-Language Framework for Reasoning of Generative Artifacts
• R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
• BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks?
• BSO: Binary Spiking Online Optimization Algorithm
• LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment
• Extreme Model Compression with Structured Sparsity at Low Precision
• Gate-level boolean evolutionary geometric attention neural networks
• Sensor Calibration Model Balancing Accuracy, Real-time, and Efficiency
• A Survey on Deep Text Hashing: Efficient Semantic Text Retrieval with Binary Representation
• LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
• Torus embeddings
• Differentiable, Bit-shifting, and Scalable Quantization without training neural network from scratch
• Efficient Cost-and-Quality Controllable Arbitrary-scale Super-resolution with Fourier Constraints
• TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge
• MC#: Mixture Compressor for Mixture-of-Experts Large Models
• Receptive Field Expanded Look-Up Tables for Vision Inference: Advancing from Low-level to High-level Tasks
• High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator
• SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
• Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compression
• Sharpness-Aware Data Generation for Zero-shot Quantization
• Spatial-Spectral Binarized Neural Network for Panchromatic and Multi-spectral Images Fusion
• HTMA-Net: Towards Multiplication-Avoiding Neural Networks via Hadamard Transform and In-Memory Computing
• PT²-LLM: Post-Training Ternarization for Large Language Models
• No One-Size-Fits-All: A Workload-Driven Characterization of Bit-Parallel vs. Bit-Serial Data Layouts for Processing-using-Memory
• Binary Autoencoder for Mechanistic Interpretability of Large Language Models
• Punching Above Precision: Small Quantized Model Distillation with Learnable Regularizer
• Memory Efficient Tabular Foundation Models
• Not All Ops Are Created Equal!
• Trained Ternary Quantization
• Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
• On construction of transfer learning for facial symmetry assessment before and after orthognathic surgery
• clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
• Deep Lookup Network
• ShaResNet: reducing residual network parameter number by sharing weights
• Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions
• Data-Augmented Quantization-Aware Knowledge Distillation
• Binary Quantization For LLMs Through Dynamic Grouping
• Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits
• Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers
• Progressive Element-wise Gradient Estimation for Neural Network Quantization
• Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA
• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
• Deep neural networks are robust to weight binarization and other non-linear distortions
• NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
• Training DNNs with Hybrid Block Floating Point
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Task complexity shapes internal representations and robustness in neural networks
• iFairy: the First 2-bit Complex LLM with All Parameters in \±1, ± i\
• Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
• Quantized neural network design under weight capacity constraint
• Bit plane [wikipedia]

Discussions

• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks [hn, 74 points, 17 comments]

Related

• XNOR-Net++: Improved Binary Neural Networks
• Binarized Neural Networks on the ImageNet Classification Task
• Back to Simplicity: How to Train Accurate BNNs from Scratch?
• Optimization of XNOR Convolution for Binary Convolutional Neural Networks on GPU
• ImageNet classification with deep convolutional neural networks