Bit Fusion: Bit-Level Dynamically Composable Architecture for\n Accelerating Deep Neural Networks

2017/12/05 by Hardik Sharma, Jongse Park, Sharma, Hardik +13 · 11 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Advanced Memory and Neural Computing #Ferroelectric and Negative Capacitance Devices

paper · pdf · doi:10.48550/arxiv.1712.01507

Abstract

Fully realizing the potential of acceleration for Deep Neural Networks (DNNs)\nrequires understanding and leveraging algorithmic properties. This paper builds\nupon the algorithmic insight that bitwidth of operations in DNNs can be reduced\nwithout compromising their classification accuracy. However, to prevent\naccuracy loss, the bitwidth varies significantly across DNNs and it may even be\nadjusted for each layer. Thus, a fixed-bitwidth accelerator would either offer\nlimited benefits to accommodate the worst-case bitwidth requirements, or lead\nto a degradation in final accuracy. To alleviate these deficiencies, this work\nintroduces dynamic bit-level fusion/decomposition as a new dimension in the\ndesign of DNN accelerators. We explore this dimension by designing Bit Fusion,\na bit-flexible accelerator, that constitutes an array of bit-level processing\nelements that dynamically fuse to match the bitwidth of individual DNN layers.\nThis flexibility in the architecture enables minimizing the computation and the\ncommunication at the finest granularity possible with no loss in accuracy. We\nevaluate the benefits of BitFusion using eight real-world feed-forward and\nrecurrent DNNs. The proposed microarchitecture is implemented in Verilog and\nsynthesized in 45 nm technology. Using the synthesis results and cycle accurate\nsimulation, we compare the benefits of Bit Fusion to two state-of-the-art DNN\naccelerators, Eyeriss and Stripes. In the same area, frequency, and process\ntechnology, BitFusion offers 3.9x speedup and 5.1x energy savings over Eyeriss.\nCompared to Stripes, BitFusion provides 2.6x speedup and 3.9x energy reduction\nat 45 nm node when BitFusion area and frequency are set to those of Stripes.\nScaling to GPU technology node of 16 nm, BitFusion almost matches the\nperformance of a 250-Watt Titan Xp, which uses 8-bit vector instructions, while\nBitFusion merely consumes 895 milliwatts of power.\n

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Gradient-based learning applied to document recognition

Cited by

• CAMformer: Associative Memory is All You Need
• Mixed-Precision Quantization for Language Models: Techniques and Prospects
• No One-Size-Fits-All: A Workload-Driven Characterization of Bit-Parallel vs. Bit-Serial Data Layouts for Processing-using-Memory
• TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge
• LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications
• MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
• BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
• Deep Learning Inference in Facebook Data Centers: Characterization,\n Performance Optimizations and Hardware Implications
• MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI
• Why Does Stochastic Gradient Descent Slow Down in Low-Precision Training?
• Of Good Demons and Bad Angels: Guaranteeing Safe Control under Finite Precision

Related

• Energy-efficient Dense DNN Acceleration with Signed Bit-slice Architecture
• Bit-Parallel Vector Composability for Neural Acceleration
• BitWave: Exploiting Column-Based Bit-Level Sparsity for Deep Learning Acceleration
• Ax-BxP: Approximate Blocked Computation for Precision-Reconfigurable Deep Neural Network Acceleration
• CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs