A pipeline for fair comparison of graph neural networks in node classification tasks

2020/12/19 by Wentao Zhao, Dalin Zhou, Zhao, Wentao +5
Computer Science · Engineering · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2012.10619

openalex publication_date 2020/12/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Graph neural networks (GNNs) have been investigated for potential applicability in multiple fields that employ graph data. However, there are no standard training settings to ensure fair comparisons among new methods, including different model architectures and data augmentation techniques. We introduce a standard, reproducible benchmark to which the same training settings can be applied for node classification. For this benchmark, we constructed 9 datasets, including both small- and medium-scale datasets from different fields, and 7 different models. We design a k-fold model assessment strategy for small datasets and a standard set of model training procedures for all datasets, enabling a standard experimental pipeline for GNNs to help ensure fair model architecture comparisons. We use node2vec and Laplacian eigenvectors to perform data augmentation to investigate how input features affect the performance of the models. We find topological information is important for node classification tasks. Increasing the number of model layers does not improve the performance except on the PATTERN and CLUSTER datasets, in which the graphs are not connected. Data augmentation is highly useful, especially using node2vec in the baseline, resulting in a substantial baseline performance improvement.

Citations

• Adam: A Method for Stochastic Optimization
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• Gated Graph Sequence Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Residual Gated Graph ConvNets
• How Powerful are Graph Neural Networks?
• Towards Sparse Hierarchical Graph Classifiers
• Pitfalls of Graph Neural Network Evaluation
• Fast Graph Representation Learning with PyTorch Geometric
• Relational Pooling for Graph Representations
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• PairNorm: Tackling Oversmoothing in GNNs
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks
• A Comprehensive Survey on Graph Neural Networks

Related

• Beyond Real-world Benchmark Datasets: An Empirical Study of Node Classification with GNNs
• When Do We Need Graph Neural Networks for Node Classification?
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Pitfalls of Graph Neural Network Evaluation
• Bag of Tricks for Node Classification with Graph Neural Networks