Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks

2021/10/26 by Pengyong Li, Jun Wang, Li, Pengyong +15 · 1 citation
Computer Science · Materials Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Artificial intelligence #Binary classification #Computational Drug Discovery Methods #Computer science #FOS: Computer and information sciences #Feature learning #Graph #Machine Learning (cs.LG) #Machine Learning in Materials Science #Machine learning #Pairwise comparison #Pattern recognition (psychology) #Simple graph #Support vector machine #Theoretical computer science #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2110.13567

published in arXiv (Cornell University) (Cornell University) · accepted by the 30th International Joint Conference on Artificial Intelligence (IJCAI-21)

openalex publication_date 2021/10/26 · openalex created_date 2021/11/08 · arxiv created 2021/12/10 · arxiv updated 2021/12/13 · openalex updated_date 2026/07/28

Abstract

Self-supervised learning has gradually emerged as a powerful technique for graph representation learning. However, transferable, generalizable, and robust representation learning on graph data still remains a challenge for pre-training graph neural networks. In this paper, we propose a simple and effective self-supervised pre-training strategy, named Pairwise Half-graph Discrimination (PHD), that explicitly pre-trains a graph neural network at graph-level. PHD is designed as a simple binary classification task to discriminate whether two half-graphs come from the same source. Experiments demonstrate that the PHD is an effective pre-training strategy that offers comparable or superior performance on 13 graph classification tasks compared with state-of-the-art strategies, and achieves notable improvements when combined with node-level strategies. Moreover, the visualization of learned representation revealed that PHD strategy indeed empowers the model to learn graph-level knowledge like the molecular scaffold. These results have established PHD as a powerful and effective self-supervised learning strategy in graph-level representation learning.

Citations

• Inductive Representation Learning on Large Graphs
• graph2vec: Learning Distributed Representations of Graphs
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• How Powerful are Graph Neural Networks?
• Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories
• Strategies for Pre-training Graph Neural Networks
• Contrastive Multi-View Representation Learning on Graphs
• DeeperGCN: All You Need to Train Deeper GCNs
• Motif-Driven Contrastive Learning of Graph Representations

Cited by

• Towards A Universal Graph Structural Encoder

Related

• Self-supervised Learning and Graph Classification under Heterophily
• Graph Self-supervised Learning with Accurate Discrepancy Learning
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations
• Strategies for Pre-training Graph Neural Networks
• Learning Graph Representation by Aggregating Subgraphs via Mutual Information Maximization