Revisiting Over-smoothing in Deep GCNs

2020/03/30 by Chaoqi Yang, Yang, Chaoqi, Ruijie Wang +7 · 8 citations
Computer Science · #Advanced Graph Neural Networks #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2003.13663

openalex publication_date 2020/03/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Oversmoothing has been assumed to be the major cause of performance drop in deep graph convolutional networks (GCNs). In this paper, we propose a new view that deep GCNs can actually learn to anti-oversmooth during training. This work interprets a standard GCN architecture as layerwise integration of a Multi-layer Perceptron (MLP) and graph regularization. We analyze and conclude that before training, the final representation of a deep GCN does over-smooth, however, it learns anti-oversmoothing during training. Based on the conclusion, the paper further designs a cheap but effective trick to improve GCN training. We verify our conclusions and evaluate the trick on three citation networks and further provide insights on neighborhood aggregation in GCNs.

Citations

• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• A tutorial on spectral clustering
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Representation Learning on Graphs with Jumping Knowledge Networks
• Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
• What graph neural networks cannot learn: depth vs width
• PairNorm: Tackling Oversmoothing in GNNs

Cited by

• How Wide and How Deep? Mitigating Over-Squashing of GNNs via Channel Capacity Constrained Estimation
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks
• DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion Prediction
• Differential-Integral Neural Operator for Long-Term Turbulence Forecasting
• Node Similarity Preserving Graph Convolutional Networks
• M4GN: Mesh-based Multi-segment Hierarchical Graph Network for Dynamic Simulations
• On Provable Benefits of Depth in Training Graph Convolutional Networks

Related

• On Provable Benefits of Depth in Training Graph Convolutional Networks
• Tackling Over-Smoothing for General Graph Convolutional Networks
• Towards a deeper GCN: Alleviate over-smoothing with iterative training and fine-tuning
• Partially Trained Graph Convolutional Networks Resist Oversmoothing
• The Oversmoothing Fallacy: A Misguided Narrative in GNN Research