Leveraging Manifold Embeddings for Enhanced Graph Transformer Representations and Learning

2025/07/09 by Jyothish, Ankit, Jannesari, Ali · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2507.07335

Abstract

Graph transformers typically embed every node in a single Euclidean space, blurring heterogeneous topologies. We prepend a lightweight Riemannian mixture-of-experts layer that routes each node to various kinds of manifold, mixture of spherical, flat, hyperbolic - best matching its local structure. These projections provide intrinsic geometric explanations to the latent space. Inserted into a state-of-the-art ensemble graph transformer, this projector lifts accuracy by up to 3% on four node-classification benchmarks. The ensemble makes sure that both euclidean and non-euclidean features are captured. Explicit, geometry-aware projection thus sharpens predictive power while making graph representations more interpretable.

Citations

• GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts
• Position: Graph Foundation Models are Already Here
• Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning
• GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts
• SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations
• NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification
• Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion
• GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings
• Rethinking Graph Transformers with Spectral Attention
• Pseudo-Riemannian Graph Convolutional Networks
• Lorentzian Graph Convolutional Networks
• Contrastive Multi-View Representation Learning on Graphs
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• Constant Curvature Graph Convolutional Networks
• Hyperbolic Graph Convolutional Neural Networks
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• Riemannian Adaptive Optimization Methods
• Graph Attention Networks
• Inductive Representation Learning on Large Graphs
• Semi-Supervised Classification with Graph Convolutional Networks

Cited by

• GRAVITY: A Controversial Graph Representation Learning for Vertex Classification

Related

• Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning
• Computationally Tractable Riemannian Manifolds for Graph Embeddings
• CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning
• FMGNN: Fused Manifold Graph Neural Network
• GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts