Beyond Low-frequency Information in Graph Convolutional Networks

2021/01/04 by Deyu Bo, Xiao Wang, Bo, Deyu +5 · 34 citations
Computer Science · Neuroscience · Engineering · #Advanced Graph Neural Networks #EEG and Brain-Computer Interfaces #Advanced Memory and Neural Computing

paper · pdf · doi:10.48550/arxiv.2101.00797

Abstract

Graph neural networks (GNNs) have been proven to be effective in various network-related tasks. Most existing GNNs usually exploit the low-frequency signals of node features, which gives rise to one fundamental question: is the low-frequency information all we need in the real world applications? In this paper, we first present an experimental investigation assessing the roles of low-frequency and high-frequency signals, where the results clearly show that exploring low-frequency signal only is distant from learning an effective node representation in different scenarios. How can we adaptively learn more information beyond low-frequency information in GNNs? A well-informed answer can help GNNs enhance the adaptability. We tackle this challenge and propose a novel Frequency Adaptation Graph Convolutional Networks (FAGCN) with a self-gating mechanism, which can adaptively integrate different signals in the process of message passing. For a deeper understanding, we theoretically analyze the roles of low-frequency signals and high-frequency signals on learning node representations, which further explains why FAGCN can perform well on different types of networks. Extensive experiments on six real-world networks validate that FAGCN not only alleviates the over-smoothing problem, but also has advantages over the state-of-the-arts.

Cited by

• Task-driven Heterophilic Graph Structure Learning
• Interpretable and Adaptive Node Classification on Heterophilic Graphs via Combinatorial Scoring and Hybrid Learning
• Unsupervised Graph Representation Learning with Complementary View Alignment
• HeAD-CP: Heterophily-Aware Diffused Conformal Prediction Sets for Graph Neural Networks
• DS-HGCN: A Dual-Stream Hypergraph Convolutional Network for Predicting Student Engagement via Social Contagion
• ATLAS: Adaptive Topology-based Learning at Scale for Homophilic and Heterophilic Graphs
• SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecasting
• Polynomial Neural Sheaf Diffusion: A Spectral Filtering Approach on Cellular Sheaves
• Gauge-Equivariant Graph Networks via Self-Interference Cancellation
• GCL-OT: Graph Contrastive Learning with Optimal Transport for Heterophilic Text-Attributed Graphs
• LaguerreNet: Advancing a Unified Solution for Heterophily and Over-smoothing with Adaptive Continuous Polynomials
• KrawtchoukNet: A Unified GNN Solution for Heterophily and Over-smoothing with Adaptive Bounded Polynomials
• Complex-Weighted Convolutional Networks: Provable Expressiveness via Complex Diffusion
• Hierarchical Frequency-Decomposition Graph Neural Networks for Road Network Representation Learning
• Beyond Fixed Depth: Adaptive Graph Neural Networks for Node Classification Under Varying Homophily
• Generalizing Graph Neural Networks on Out-Of-Distribution Graphs
• Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
• Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
• Unifying and Enhancing Graph Transformers via a Hierarchical Mask Framework
• HeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relations
• GraphUniverse: Synthetic Graph Generation for Evaluating Inductive Generalization
• Bridging Computational Social Science and Deep Learning: Cultural Dissemination-Inspired Graph Neural Networks
• Gumbel-MPNN: Graph Rewiring with Gumbel-Softmax
• Graph Homophily Booster: Rethinking the Role of Discrete Features on Heterophilic Graphs
• Learning from Heterophilic Graphs: A Spectral Theory Perspective on the Impact of Self-Loops and Parallel Edges
• Why does your graph neural network fail on some graphs? Insights from exact generalisation error
• Graph Alignment via Dual-Pass Spectral Encoding and Latent Space Communication
• InfraredGP: Efficient Graph Partitioning via Spectral Graph Neural Networks with Negative Corrections
• Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
• TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows
• Adaptive Riemannian Graph Neural Networks
• Sheaf Graph Neural Networks via PAC-Bayes Spectral Optimization
• MPFSR-Enhanced GNNs: Spectral Graph Neural Networks Enhancement Through Learnable Multiple-Parameter Graph Fractional Fourier Transforms
• Simple Truncated SVD based Model for Node Classification on Heterophilic Graphs
• GraphTorque: Torque-Driven Rewiring Graph Neural Network

Related

• Node-Variant Graph Filters in Graph Neural Networks
• Graph Neural Networks: Architectures, Stability and Transferability
• Quantifying the Optimization and Generalization Advantages of Graph Neural Networks Over Multilayer Perceptrons
• Improving Your Graph Neural Networks: A High-Frequency Booster
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks