PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs

2025/03/31 by Malekar, Jinendra, Chandarana, Peyton, Amin, Md Hasibul +2 · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Hardware Architecture (cs.AR)

paper · doi:10.48550/arxiv.2504.01994

Abstract

In this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively.

Cited by

• Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM

Related

• CompAir: Synergizing Complementary PIMs and In-Transit NoC Computation for Efficient LLM Acceleration
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• PIM-AI: A Novel Architecture for High-Efficiency LLM Inference
• Matmul or No Matmul in the Era of 1-bit LLMs
• LEAP: LLM Inference on Scalable PIM-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism