Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing
2025/06/06 by Taha, Diaaeldin, Chapman, James, Eidi, Marzieh +2
#FOS: Computer and information sciences #I.5.1 #Machine Learning (cs.LG) #Machine Learning (stat.ML)
paper · doi:10.48550/arxiv.2506.06582
Abstract
Topological deep learning (TDL) has emerged as a powerful tool for modeling higher-order interactions in relational data. However, phenomena such as oversquashing in topological message-passing remain understudied and lack theoretical analysis. We propose a unifying axiomatic framework that bridges graph and topological message-passing by viewing simplicial and cellular complexes and their message-passing schemes through the lens of relational structures. This approach extends graph-theoretic results and algorithms to higher-order structures, facilitating the analysis and mitigation of oversquashing in topological message-passing networks. Through theoretical analysis and empirical studies on simplicial networks, we demonstrate the potential of this framework to advance TDL.
Citations
- TopoTune : A Framework for Generalized Combinatorial Complex Neural Networks
- Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
- The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited
- How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing
- Spectral Graph Pruning Against Over-Squashing and Over-Smoothing
- Position: Topological Deep Learning is the New Frontier for Relational Learning
- Link Prediction with Relational Hypergraphs
- Design Your Own Universe: A Physics-Informed Agnostic Method for Enhancing Graph Neural Networks
- Relational Deep Learning: Graph Representation Learning on Relational Databases
- DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
- Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges
- Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature
- A Survey on Hypergraph Representation Learning
- A Survey on Oversmoothing in Graph Neural Networks
- Understanding Oversquashing in GNNs through the Lens of Effective Resistance
- On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology
- Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature
- FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
- Ollivier curvature, betweenness centrality and average distance
- Cell Attention Networks
- Oversquashing in GNNs through the lens of information contraction and graph expansion
- Simplicial Attention Networks
- Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs
- Understanding over-squashing and bottlenecks on graphs via curvature
- Beltrami Flow and Neural Diffusion on Graphs
- Weisfeiler and Lehman Go Cellular: CW Networks
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Knowledge Hypergraph Embedding Meets Relational Algebra
- Learning Parametrised Graph Shift Operators
- A Generalization of Transformer Networks to Graphs
- Simplicial Neural Networks
- Cell Complex Neural Networks
- TUDataset: A collection of benchmark datasets for learning with graphs
- Array programming with NumPy
- On the Bottleneck of Graph Neural Networks and its Practical Implications
- Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Composition-based Multi-Relational Graph Convolutional Networks
- Stability of Graph Neural Networks to Relative Perturbations
- Ollivier Ricci Curvature of Directed Hypergraphs
- Provably Powerful Graph Networks
- Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
- Fast Graph Representation Learning with PyTorch Geometric
- Connecting the Dots: Identifying Network Structure via Graph Signal Processing
- Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
- How Powerful are Graph Neural Networks?
- Tune: A Research Platform for Distributed Model Selection and Training
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
- Ray: A Distributed Framework for Emerging AI Applications
- Graph Attention Networks
- Spectrally-normalized margin bounds for neural networks
- Modeling Relational Data with Graph Convolutional Networks
- Semi-Supervised Classification with Graph Convolutional Networks
- Matplotlib: A 2D Graphics Environment
- Topological Deep Learning: Going Beyond Graph Data
Related