NX-CGRA: A Programmable Hardware Accelerator for Core Transformer Algorithms on Edge Devices

2025/11/21 by Prasad, Rohit
#FOS: Computer and information sciences #Hardware Architecture (cs.AR)

paper · doi:10.48550/arxiv.2511.17235

Abstract

The increasing diversity and complexity of transformer workloads at the edge present significant challenges in balancing performance, energy efficiency, and architectural flexibility. This paper introduces NX-CGRA, a programmable hardware accelerator designed to support a range of transformer inference algorithms, including both linear and non-linear functions. Unlike fixed-function accelerators optimized for narrow use cases, NX-CGRA employs a coarse-grained reconfigurable array (CGRA) architecture with software-driven programmability, enabling efficient execution across varied kernel patterns. The architecture is evaluated using representative benchmarks derived from real-world transformer models, demonstrating high overall efficiency and favorable energy-area tradeoffs across different classes of operations. These results indicate the potential of NX-CGRA as a scalable and adaptable hardware solution for edge transformer deployment under constrained power and silicon budgets.

Citations

• TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings
• FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization
• RedMule: A Mixed-Precision Matrix-Matrix Operation Engine for Flexible and Energy-Efficient On-Chip Linear Algebra and TinyML Training Acceleration
• Rethinking Vision Transformers for MobileNet Size and Speed
• TinyViT: Fast Pretraining Distillation for Small Vision Transformers
• A Survey of Transformers
• MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
• A³: Accelerating Attention Mechanisms in Neural Networks with\n Approximation
• Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via\n Full-Stack Integration
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
• TinyBERT: Distilling BERT for Natural Language Understanding
• Caffe con Troll: Shallow Ideas to Speed Up Deep Learning
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• An ultra-low-power CGRA for accelerating Transformers at the edge
• EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms
• CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference
• COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inference
• EdgeFlex-Transformer: Transformer Inference for Edge Devices