Optimizing Block-Sparse Matrix Multiplications on CUDA with TVM

2020/07/26 by Gu, Zijing
#Distributed #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Mathematical Software (cs.MS) #Numerical Analysis (math.NA) #Parallel #and Cluster Computing (cs.DC)

paper · doi:10.48550/arxiv.2007.13055

Abstract

We implemented and optimized matrix multiplications between dense and block-sparse matrices on CUDA. We leveraged TVM, a deep learning compiler, to explore the schedule space of the operation and generate efficient CUDA code. With the automatic parameter tuning in TVM, our cross-thread reduction based implementation achieved competitive or better performance compared with other state-of-the-art frameworks.

Related

• CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
• Compiler-Level Matrix Multiplication Optimization for Deep Learning
• Stream-K: Work-centric Parallel Decomposition for Dense Matrix-Matrix Multiplication on the GPU
• NM-SpMM: Accelerating Matrix Multiplication Using N:M Sparsity with GPGPU
• DBCSR: A Library for Dense Matrix Multiplications on Distributed GPU-Accelerated Systems