2024/11/08 by Nurlan Nazaraliyev, Nazaraliyev, Nurlan, Elaheh Sadredini +3
Computer Science · #Distributed #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Operating Systems (cs.OS) #Parallel #Parallel Computing and Optimization Techniques #Performance (cs.PF) #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2411.05309
openalex publication_date 2024/11/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Graphics Processing Units (GPUs) leverage massive parallelism and large memory bandwidth to support high-performance computing applications, such as multimedia rendering, crypto-mining, deep learning, and natural language processing. These applications require models and datasets that are getting bigger in size and currently challenge the memory capacity of a single GPU, causing substantial performance overheads. To address this problem, a programmer has to partition the data and manually transfer data in and out of the GPU. This approach requires programmers to carefully tune their applications and can be impractical for workloads with irregular access patterns, such as deep learning, recommender systems, and graph applications. To ease programmability, programming abstractions such as unified virtual memory (UVM) can be used, creating a virtually unified memory space across the whole system and transparently moving the data on demand as it is accessed. However, UVM brings in the overhead of the OS involvement and inefficiencies due to generating many transfer requests especially when the GPU memory is oversubscribed. This paper proposes GPUVM, a GPU memory management system that uses an RDMA-capable network device to construct a virtual memory system without involving the CPU/OS. GPUVM enables on-demand paging for GPU applications and relies on GPU threads for memory management and page migration. Since CPU chipsets do not support GPU-driven memory management, we use a network interface card to facilitate transparent page migration from/to the GPU. GPUVM achieves performance up to 4x higher than UVM for latency-bound applications while providing accessible programming abstractions that do not require the users to manage memory transfers directly.