Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors

2022/01/25 by Shurui Li, Puneet Gupta, Li, Shurui +1 · 1 citation
Computer Science · #Parallel Computing and Optimization Techniques #Neural Networks and Applications #Numerical Methods and Algorithms

paper · pdf · doi:10.48550/arxiv.2201.11651

Abstract

Applications of neural networks on edge systems have proliferated in recent years but the ever-increasing model size makes neural networks not able to deploy on resource-constrained microcontrollers efficiently. We propose bit-serial weight pools, an end-to-end framework that includes network compression and acceleration of arbitrary sub-byte precision. The framework can achieve up to 8x compression compared to 8-bit networks by sharing a pool of weights across the entire network. We further propose a bit-serial lookup based software implementation that allows runtime-bitwidth tradeoff and is able to achieve more than 2.8x speedup and 7.5x storage compression compared to 8-bit weight pool networks, with less than 1% accuracy drop.

Cited by

• Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation

Related

• CIMPool: Scalable Neural Network Acceleration for Compute-In-Memory using Weight Pools
• Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity
• SWIS -- Shared Weight bIt Sparsity for Efficient Neural Network\n Acceleration
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Self-Compressing Neural Networks