XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks

2016/03/16 by Mohammad Rastegari, Rastegari, Mohammad, Vicente Ordonez +6 · 1 voice · 132 citations
Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Algorithm #Arithmetic #Artificial intelligence #Artificial neural network #Binary classification #Binary number #Computer science #Convolutional neural network #Domain Adaptation and Few-Shot Learning #Logic gate #Mathematics #Pattern recognition (psychology) #Support vector machine #XNOR gate #cs.CV

paper · pdf · doi:10.48550/arxiv.1603.05279

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2016/03/16 · arxiv created 2016/08/02 · arxiv updated 2016/08/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose two efficient approximations to standard convolutional neural\nnetworks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks,\nthe filters are approximated with binary values resulting in 32x memory saving.\nIn XNOR-Networks, both the filters and the input to convolutional layers are\nbinary. XNOR-Networks approximate convolutions using primarily binary\noperations. This results in 58x faster convolutional operations and 32x memory\nsavings. XNOR-Nets offer the possibility of running state-of-the-art networks\non CPUs (rather than GPUs) in real-time. Our binary networks are simple,\naccurate, efficient, and work on challenging visual tasks. We evaluate our\napproach on the ImageNet classification task. The classification accuracy with\na Binary-Weight-Network version of AlexNet is only 2.9% less than the\nfull-precision AlexNet (in top-1 measure). We compare our method with recent\nnetwork binarization methods, BinaryConnect and BinaryNets, and outperform\nthese methods by large margins on ImageNet, more than 16% in top-1 accuracy.\n

Citations

• Big Neural Networks Waste Capacity
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Deep Residual Learning for Image Recognition
• Predicting Parameters in Deep Learning

Cited by

• Task-Oriented Communication with Hybrid-Precision Models
• Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
• 1 bit is all we need: binary normalized neural networks
• X-pSRAM: A Photonic SRAM with Embedded XOR Logic for Ultra-Fast In-Memory Computing
• A Principled Bayesian Framework for Training Binary and Spiking Neural Networks
• Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
• Binary Neural Network Implementation for Handwritten Digit Recognition on FPGA
• Hardware Software Optimizations for Fast Model Recovery on Reconfigurable Architectures
• BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training
• INSIGHT: An Interpretable Neural Vision-Language Framework for Reasoning of Generative Artifacts
• R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
• BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks?
• BSO: Binary Spiking Online Optimization Algorithm
• LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment
• Extreme Model Compression with Structured Sparsity at Low Precision
• Gate-level boolean evolutionary geometric attention neural networks
• Sensor Calibration Model Balancing Accuracy, Real-time, and Efficiency
• A Survey on Deep Text Hashing: Efficient Semantic Text Retrieval with Binary Representation
• LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
• Torus embeddings
• Differentiable, Bit-shifting, and Scalable Quantization without training neural network from scratch
• Efficient Cost-and-Quality Controllable Arbitrary-scale Super-resolution with Fourier Constraints
• TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge
• MC#: Mixture Compressor for Mixture-of-Experts Large Models
• Receptive Field Expanded Look-Up Tables for Vision Inference: Advancing from Low-level to High-level Tasks
• High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator
• SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
• Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compression
• Sharpness-Aware Data Generation for Zero-shot Quantization
• Spatial-Spectral Binarized Neural Network for Panchromatic and Multi-spectral Images Fusion
• HTMA-Net: Towards Multiplication-Avoiding Neural Networks via Hadamard Transform and In-Memory Computing
• PT²-LLM: Post-Training Ternarization for Large Language Models
• No One-Size-Fits-All: A Workload-Driven Characterization of Bit-Parallel vs. Bit-Serial Data Layouts for Processing-using-Memory
• Binary Autoencoder for Mechanistic Interpretability of Large Language Models
• Punching Above Precision: Small Quantized Model Distillation with Learnable Regularizer
• Memory Efficient Tabular Foundation Models
• Not All Ops Are Created Equal!
• Trained Ternary Quantization
• Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
• On construction of transfer learning for facial symmetry assessment before and after orthognathic surgery
• clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
• Deep Lookup Network
• ShaResNet: reducing residual network parameter number by sharing weights
• Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions
• Data-Augmented Quantization-Aware Knowledge Distillation
• Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping
• Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits
• Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers
• Progressive Element-wise Gradient Estimation for Neural Network Quantization
• Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA
• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
• Deep neural networks are robust to weight binarization and other non-linear distortions
• NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
• Training DNNs with Hybrid Block Floating Point
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Task complexity shapes internal representations and robustness in neural networks
• iFairy: the First 2-bit Complex LLM with All Parameters in \±1, ± i\
• Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
• Quantized neural network design under weight capacity constraint
• Exploration of Low Numeric Precision Deep Learning Inference Using Intel FPGAs
• A Lower Bound for the Number of Linear Regions of Ternary ReLU Regression Neural Networks
• Delta Networks for Optimized Recurrent Network Computation
• Towards Evolutional Compression
• FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
• Scientific Calculator for Designing Trojan Detectors in Neural Networks
• A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks
• Understanding the Energy and Precision Requirements for Online Learning
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression
• A Survey on Deep Learning Methods for Robot Vision
• A Method for Optimizing Connections in Differentiable Logic Gate Networks
• Secure and Storage-Efficient Deep Learning Models for Edge AI Using Automatic Weight Generation
• Model Compression using Progressive Channel Pruning
• Structured Convolution Matrices for Energy-efficient Deep learning
• BiVM: Accurate Binarized Neural Network for Efficient Video Matting
• Information-Bottleneck Driven Binary Neural Network for Change Detection
• HarDNet: A Low Memory Traffic Network
• High-Fidelity Differential-information Driven Binary Vision Transformer
• P²U: Progressive Precision Update For Efficient Model Distribution
• ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks
• Efficient Certified Reasoning for Binarized Neural Networks
• Can Argus Judge Them All? Comparing VLMs Across Domains
• BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
• MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Models
• Compression Aware Certified Training
• Advances in Small-Footprint Keyword Spotting: A Comprehensive Review of Efficient Models and Algorithms
• Streamlined Deployment for Quantized Neural Networks
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• In-memory Implementation of On-chip Trainable and Scalable ANN for AI/ML Applications
• BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing
• Learning Binarized Representations with Pseudo-positive Sample Enhancement for Efficient Graph Collaborative Filtering
• PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs
• Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1
• LittleBit: Ultra Low-Bit Quantization via Latent Factorization
• Training Bit Fully Convolutional Network for Fast Semantic Segmentation
• Learning Interpretable Differentiable Logic Networks for Tabular Regression
• SineLoRAΔ: Sine-Activated Delta Compression
• Highly Efficient and Effective LLMs with Multi-Boolean Architectures
• Efficient Continual Learning in Keyword Spotting using Binary Neural Networks
• Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision
• Spiking Neural Networks Need High Frequency Information
• BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
• Beyond Discreteness: Finite-Sample Analysis of Straight-Through Estimator for Quantization
• Searching for Low-Bit Weights in Quantized Neural Networks
• Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantization
• An Overview of Arithmetic Adaptations for Inference of Convolutional Neural Networks on Re-configurable Hardware
• Coarse and fine-grained automatic cropping deep convolutional neural network
• Energy-Aware Deep Learning on Resource-Constrained Hardware
• Addition is almost all you need: Compressing large language models with double binary factorization
• FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review
• Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
• CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
• Machine Learning Models that Remember Too Much
• Practical Boolean Backpropagation
• Optimizing Deep Neural Networks using Safety-Guided Self Compression
• Fast Video Classification via Adaptive Cascading of Deep Models
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Energy-Efficient Neuromorphic Computing for Edge AI: A Framework with Adaptive Spiking Neural Networks and Hardware-Aware Optimization
• Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
• Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration
• Silenzio: Secure Non-Interactive Outsourced MLP Training
• Binarized Neural Networks on the ImageNet Classification Task
• BackSlash: Rate Constrained Optimized Training of Large Language Models
• Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions
• Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation
• One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantization
• ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Accelerators
• Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training
• AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture
• Compound and Parallel Modes of Tropical Convolutional Neural Networks
• Bit plane [wikipedia]

Discussions

• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks [hn, 74 points, 17 comments]

Related

• XNOR-Net++: Improved Binary Neural Networks
• Binarized Neural Networks on the ImageNet Classification Task
• Back to Simplicity: How to Train Accurate BNNs from Scratch?
• Optimization of XNOR Convolution for Binary Convolutional Neural Networks on GPU
• ImageNet classification with deep convolutional neural networks