2022/07/28 by Rohan Yadav, Yadav, Rohan, Alex Aiken +4 · 4 citations
Computer Science · #Advanced Neural Network Applications #Distributed #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Parallel #Parallel Computing and Optimization Techniques #Programming Languages (cs.PL) #and Cluster Computing (cs.DC) #cs.DC #cs.PL
paper · pdf · doi:10.48550/arxiv.2207.13901
arxiv created 2022/07/28 · openalex publication_date 2022/07/28 · arxiv updated 2022/07/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce SpDISTAL, a compiler for sparse tensor algebra that targets distributed systems. SpDISTAL combines separate descriptions of tensor algebra expressions, sparse data structures, data distribution, and computation distribution. Thus, it enables distributed execution of sparse tensor algebra expressions with a wide variety of sparse data structures and data distributions. SpDISTAL is implemented as a C++ library that targets a distributed task-based runtime system and can generate code for nodes with both multi-core CPUs and multiple GPUs. SpDISTAL generates distributed code that achieves performance competitive with hand-written distributed functions for specific sparse tensor algebra expressions and that outperforms general interpretation-based systems by one to two orders of magnitude.