When Does Self-Supervision Help Graph Convolutional Networks?

2020/06/16 by Yuning You, You, Yuning, Tianlong Chen +4 · 3 citations
Computer Science · #Advanced Graph Neural Networks #Domain Adaptation and Few-Shot Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2006.09136

openalex publication_date 2020/06/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Self-supervision as an emerging technique has been employed to train convolutional neural networks (CNNs) for more transferrable, generalizable, and robust representation learning of images. Its introduction to graph convolutional networks (GCNs) operating on graph data is however rarely explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into GCNs. We first elaborate three mechanisms to incorporate self-supervision into GCNs, analyze the limitations of pretraining & finetuning and self-training, and proceed to focus on multi-task learning. Moreover, we propose to investigate three novel self-supervised learning tasks for GCNs with theoretical rationales and numerical comparisons. Lastly, we further integrate multi-task self-supervision into graph adversarial training. Our results show that, with properly designed task forms and incorporation mechanisms, self-supervision benefits GCNs in gaining more generalizability and robustness. Our codes are available at https://github.com/Shen-Lab/SS-GCNs.

Citations

• The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains
• Semi-Supervised Classification with Graph Convolutional Networks
• Adversarial Attack on Graph Structured Data
• How Powerful are Graph Neural Networks?
• Selfie: Self-supervised Pretraining for Image Embedding
• When Does Self-supervision Improve Few-shot Learning?
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning

Cited by

• 3D-GSRD: 3D Molecular Graph Auto-Encoder with Selective Re-mask Decoding
• Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation
• Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics

Related

• A critical analysis of self-supervision, or what we can learn from a single image
• Graph-Based Neural Network Models with Multiple Self-Supervised\n Auxiliary Tasks
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-supervision
• Multi-task Self-distillation for Graph-based Semi-Supervised Learning