2023/11/29 by Yueqiao Zhong, Zhong, Yuchen, Guangming Sheng +9
Computer Science · Medicine · #Advanced Graph Neural Networks #Dementia and Cognitive Impairment Research #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Recommender Systems and Techniques #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2311.17410
openalex publication_date 2023/11/29 · openalex created_date 2023/12/02 · openalex updated_date 2026/07/28
Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are dynamic and contain time domain information. We introduce GNNFlow, a distributed framework that enables efficient continuous temporal graph representation learning on dynamic graphs on multi-GPU machines. GNNFlow introduces an adaptive time-indexed block-based data structure that effectively balances memory usage with graph update and sampling operation efficiency. It features a hybrid GPU-CPU graph data placement for rapid GPU-based temporal neighborhood sampling and kernel optimizations for enhanced sampling processes. A dynamic GPU cache for node and edge features is developed to maximize cache hit rates through reuse and restoration strategies. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. We implement GNNFlow based on DGL and PyTorch. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.