A streamlined AlexNet with 1-bit weights, 2-bit activations

2017/09/01 by Yaman Umuroglu, Umuroglu, Yaman, Magnus Jahre +1 · 3 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices

paper · pdf · doi:10.48550/arxiv.1709.04060

openalex publication_date 2017/09/01 · openalex created_date 2017/09/25 · openalex updated_date 2026/07/28

Abstract

Running Deep Neural Network (DNN) models on devices with limited computational capability is a challenge due to large compute and memory requirements. Quantized Neural Networks (QNNs) have emerged as a potential solution to this problem, promising to offer most of the DNN accuracy benefits with much lower computational cost. However, harvesting these benefits on existing mobile CPUs is a challenge since operations on highly quantized datatypes are not natively supported in most instruction set architectures (ISAs). In this work, we first describe a streamlining flow to convert all QNN inference operations to integer ones. Afterwards, we provide techniques based on processing one bit position at a time (bit-serial) to show how QNNs can be efficiently deployed using common bitwise operations. We demonstrate the potential of QNNs on mobile CPUs with microbenchmarks and on a quantized AlexNet, which is 3.5x faster than an optimized 8-bit baseline. Our bit-serial matrix multiplication library is available on GitHub at https://git.io/vhshn

Citations

• Binarized Neural Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks

Cited by

• SIRA: Scaled-Integer Range Analysis for Optimizing FPGA Dataflow Neural Network Accelerators
• Red grape detection with accelerated artificial neural networks in the FPGA's programmable logic
• FINN-GL: Generalized Mixed-Precision Extensions for FPGA-Accelerated LSTMs

Related

• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• XpulpNN: Enabling Energy Efficient and Flexible Inference of Quantized Neural Network on RISC-V based IoT End Nodes
• A Survey on Methods and Theories of Quantized Neural Networks
• Quantization and Training of Neural Networks for Efficient\n Integer-Arithmetic-Only Inference