2019/03/06 by Esha Choukse, Choukse, Esha, Michael Sullivan +11 · 3 citations
Computer Science · #Advanced Data Storage Technologies #Cloud Computing and Resource Management #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Parallel Computing and Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1903.02596
openalex publication_date 2019/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
GPUs offer orders-of-magnitude higher memory bandwidth than traditional\nCPU-only systems. However, GPU device memory tends to be relatively small and\nthe memory capacity can not be increased by the user. This paper describes\nBuddy Compression, a scheme to increase both the effective GPU memory capacity\nand bandwidth while avoiding the downsides of conventional memory-expanding\nstrategies. Buddy Compression compresses GPU memory, splitting each compressed\nmemory entry between high-speed device memory and a slower-but-larger\ndisaggregated memory pool (or system memory). Highly-compressible memory\nentries can thus be accessed completely from device memory, while\nincompressible entries source their data using both on and off-device accesses.\nIncreasing the effective GPU memory capacity enables us to run\nlarger-memory-footprint HPC workloads and larger batch-sizes or models for DL\nworkloads than current memory capacities would allow. We show that our solution\nachieves an average compression ratio of 2.2x on HPC workloads and 1.5x on DL\nworkloads, with a slowdown of just 1~2%.\n