2023/11/13 by Vasileios Titopoulos, Titopoulos, V., Kostas Alexandridis +7 · 1 citation
Computer Science · Mathematics · #Advanced Neural Network Applications #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning and ELM #Tensor decomposition and applications
paper · pdf · doi:10.48550/arxiv.2311.07241
openalex publication_date 2023/11/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Structured sparsity has been proposed as an efficient way to prune the complexity of modern Machine Learning (ML) applications and to simplify the handling of sparse data in hardware. The acceleration of ML models - for both training and inference - relies primarily on equivalent matrix multiplications that can be executed efficiently on vector processors or custom matrix engines. The goal of this work is to incorporate the simplicity of structured sparsity into vector execution, thereby accelerating the corresponding matrix multiplications. Toward this objective, a new vector index-multiply-accumulate instruction is proposed, which enables the implementation of lowcost indirect reads from the vector register file. This reduces unnecessary memory traffic and increases data locality. The proposed new instruction was integrated in a decoupled RISCV vector processor with negligible hardware cost. Extensive evaluation demonstrates significant speedups of 1.80x-2.14x, as compared to state-of-the-art vectorized kernels, when executing layers of varying sparsity from state-of-the-art Convolutional Neural Networks (CNNs).