Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

2021/06/16 by Geng Yuan, Zhiheng Liao, Yuan, Geng +25 · 1 citation
Computer Science · Engineering · Mathematics · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Column (typography) #Computer engineering #Computer network #Computer science #Crossbar switch #Distributed computing #Domain (mathematical analysis) #Edge device #Engineering #Enhanced Data Rates for GSM Evolution #FOS: Computer and information sciences #Fault tolerance #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Mathematics #Multiplication (music) #Parallel computing #Performance (cs.PF) #Pruning #Resistive random-access memory #cs.AI #cs.LG #cs.PF

paper · pdf · doi:10.48550/arxiv.2106.09166

published in arXiv (Cornell University) (Cornell University) · In Proceedings of the 22nd International Symposium on Quality Electronic Design (ISQED), 2021

openalex publication_date 2021/06/16 · arxiv created 2021/06/18 · arxiv updated 2021/06/22 · openalex created_date 2021/06/22 · openalex updated_date 2026/08/06

Abstract

Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication -- the intensive and key computation in deep neural networks (DNNs). However, hardware failure, such as stuck-at-fault defects, is one of the main concerns that impedes the ReRAM devices to be a feasible solution for real implementations. The existing solutions to address this issue usually require an optimization to be conducted for each individual device, which is impractical for mass-produced products (e.g., IoT devices). In this paper, we rethink the value of weight pruning in ReRAM-based DNN design from the perspective of model fault tolerance. And a differential mapping scheme is proposed to improve the fault tolerance under a high stuck-on fault rate. Our method can tolerate almost an order of magnitude higher failure rate than the traditional two-column method in representative DNN tasks. More importantly, our method does not require extra hardware cost compared to the traditional two-column mapping scheme. The improvement is universal and does not require the optimization process for each individual device.

Citations

• Memristor-The missing circuit element
• Trained Ternary Quantization
• Neuro-memristive Circuits for Edge Computing: A review
• ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers
• TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey

Cited by

• EPSILON: Adaptive Fault Mitigation in Approximate Deep Neural Network using Statistical Signatures

Related

• Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM
• X-CHANGR: Changing Memristive Crossbar Mapping for Mitigating Line-Resistance Induced Accuracy Degradation in Deep Neural Networks
• VECOM: Variation Resilient Encoding and Offset Compensation Schemes for Reliable ReRAM Based DNN Accelerator
• The Reliability Issue in ReRam-based CIM Architecture for SNN: A Survey
• Exploring Bit-Slice Sparsity in Deep Neural Networks for Efficient ReRAM-Based Deployment