2023/12/20 by Shiwei Liu, Chen Mu, Hao Jiang +6 · 2 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Ferroelectric and Negative Capacitance Devices #Machine Learning and ELM
paper · doi:10.1109/tvlsi.2023.3337777
openalex publication_date 2023/12/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30
Self-attention-based transformers have outperformed recurrent and convolutional neural networks (RNN/ CNNs) in many applications. Despite the effectiveness, calculating self-attention is prohibitively costly due to quadratic computation and memory requirements. To solve this challenge, this article proposes a hybrid analog-ReRAM and digital-SRAM in-memory computing accelerator (HARDSEA), a computing-in-memory (CIM) accelerator supporting self-attention in transformer applications. To trade off between energy efficiency and algorithm accuracy, HARDSEA features an algorithm-architecture-circuit codesign. A product-quantization-based scheme dynamically facilitates self-attention sparsity by predicting lightweight token relevance. A hybrid in-memory computing architecture employs both high-efficiency analog ReRAM-CIM and high-precision digital SRAM-CIM to implement the proposed new scheme. The ReRAM-CIM, whose precision is sensitive to circuit nonidealities, takes charge of token relevance prediction where only computing monotonicity is demanded. The SRAM-CIM, utilized for exact sparse attention computing, is reorganized as an on-memory-boundary computing scheme, thus adapting to irregular sparsity patterns. In addition, we propose a time-domain winner-take-all (WTA) circuit to replace the expensive ADCs in ReRAM-CIM macros. Experimental results show that HARDSEA prunes BERT and GPT-2 models to 12%–33% sparsity without accuracy loss, achieving13.5× –28.5× speedup and291.6× –1894.3× energy efficiency over GPU. Compared to state-of-the-art transformer accelerators, HARDSEA has1.2× –14.9× better energy efficiency at the same level of throughput.