Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training

2021/04/19 by Chuxiong Sun, Sun, Chuxiong, Hongming Gu +3 · 20 citations
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computer science #Database #FOS: Computer and information sciences #Geography #Graph #Machine Learning (cs.LG) #Machine learning #Scalability #Text and Document Classification Technologies #Theoretical computer science #Topic Modeling #Training (meteorology) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2104.09376

published in arXiv (Cornell University) (Cornell University) · 23 pages, 13 figures, fixed typos, add authors

arxiv created 2021/07/01 · arxiv updated 2021/07/02

Abstract

It is hard to directly implement Graph Neural Networks (GNNs) on large scaled graphs. Besides of existed neighbor sampling techniques, scalable methods decoupling graph convolutions and other learnable transformations into preprocessing and post classifier allow normal minibatch training. By replacing redundant concatenation operation with attention mechanism in SIGN, we propose Scalable and Adaptive Graph Neural Networks (SAGN). SAGN can adaptively gather neighborhood information among different hops. To further improve scalable models on semi-supervised learning tasks, we propose Self-Label-Enhance (SLE) framework combining self-training approach and label propagation in depth. We add base model with a scalable node label module. Then we iteratively train models and enhance train set in several stages. To generate input of node label module, we directly apply label propagation based on one-hot encoded label vectors without inner random masking. We find out that empirically the label leakage has been effectively alleviated after graph convolutions. The hard pseudo labels in enhanced train set participate in label propagation with true labels. Experiments on both inductive and transductive datasets demonstrate that, compared with other sampling-based and sampling-free methods, SAGN achieves better or comparable results and SLE can further improve performance.

Cited by

• ATLAS: Adaptive Topology-based Learning at Scale for Homophilic and Heterophilic Graphs
• VISAGNN: Versatile Staleness-Aware Efficient Training on Large-Scale Graphs
• DyC-STG: Dynamic Causal Spatio-Temporal Graph Network for Real-time Data Credibility Analysis in IoT
• Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
• H³GNNs: Harmonizing Heterophily and Homophily in GNNs via Joint Structural Node Encoding and Self-Supervised Learning
• Federated Prototype Graph Learning

Related

• Neural Graph Machines: Learning Neural Networks Using Graphs
• Adaptive Label Smoothing To Regularize Large-Scale Graph Training
• Label-Enhanced Graph Neural Network for Semi-supervised Node Classification
• On the Equivalence of Decoupled Graph Convolution Network and Label Propagation
• Training a Label-Noise-Resistant GNN with Reduced Complexity