Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms

2025/10/15 by Tong Qiao, Qiao, Tong, Ao Zhou +11
Computer Science · #Advanced Graph Neural Networks #Big Data and Digital Economy #Distributed #FOS: Computer and information sciences #Graph Theory and Algorithms #Parallel #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2511.07421

openalex publication_date 2025/10/15 · openalex created_date 2025/11/13 · openalex updated_date 2026/07/28

Abstract

Graph Neural Networks (GNNs) have been widely adopted due to their strong performance. However, GNN training often relies on expensive, high-performance computing platforms, limiting accessibility for many tasks. Profiling of representative GNN workloads indicates that substantial efficiency gains are possible on resource-constrained devices by fully exploiting available resources. This paper introduces A3GNN, a framework for affordable, adaptive, and automatic GNN training on heterogeneous CPU-GPU platforms. It improves resource usage through locality-aware sampling and fine-grained parallelism scheduling. Moreover, it leverages reinforcement learning to explore the design space and achieve pareto-optimal trade-offs among throughput, memory footprint, and accuracy. Experiments show that A3GNN can bridge the performance gap, allowing seven Nvidia 2080Ti GPUs to outperform two A100 GPUs by up to 1.8X in throughput with minimal accuracy loss.

Citations

• GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration
• Architectural Implications of GNN Aggregation Programming Abstractions
• Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness
• Hardware-Aware Graph Neural Network Automated Design for Edge Computing Platforms
• Scalable Graph Convolutional Network Training on Distributed-Memory Systems
• BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling
• BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Fast Graph Representation Learning with PyTorch Geometric

Related

• A Unified CPU-GPU Protocol for GNN Training
• NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments
• Characterizing and Understanding Distributed GNN Training on GPUs
• Characterizing and Understanding HGNN Training on GPUs
• CaPGNN: Optimizing Parallel Graph Neural Network Training with Joint Caching and Resource-Aware Graph Partitioning