A Deep Learning Inference Scheme Based on Pipelined Matrix Multiplication Acceleration Design and Non-uniform Quantization

2021/10/10 by Zhang, Yuyang, Leung, Dik Hin, Guo, Min +6
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2110.04861

Abstract

Matrix multiplication is the bedrock in Deep Learning inference application. When it comes to hardware acceleration on edge computing devices, matrix multiplication often takes up a great majority of the time. To achieve better performance in edge computing, we introduce a low-power Multi-layer Perceptron (MLP) accelerator based on a pipelined matrix multiplication scheme and a nonuniform quantization methodology. The implementation is running on Field-programmable Gate Array (FPGA) devices and tested its performance on handwritten digit classification and Q-learning tasks. Results show that our method can achieve better performance with fewer power consumption.

Related

• Low- and Mixed-Precision Inference Accelerators
• MSP: An FPGA-Specific Mixed-Scheme, Multi-Precision Deep Neural Network Quantization Framework
• Multiplierless and Sparse Machine Learning based on Margin Propagation Networks
• On-FPGA Training with Ultra Memory Reduction: A Low-Precision Tensor Method
• Benchmarking Edge AI Platforms for High-Performance ML Inference