Node Identifiers: Compact, Discrete Representations for Efficient Graph Learning

2024/05/26 by Yuankai Luo, Luo, Yuankai, Qijiong Liu +6 · 6 citations
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2405.16435

openalex publication_date 2024/05/26 · openalex created_date 2024/05/29 · openalex updated_date 2026/07/28

Abstract

We present a novel end-to-end framework that generates highly compact (typically 6-15 dimensions), discrete (int4 type), and interpretable node representations, termed node identifiers (node IDs), to tackle inference challenges on large-scale graphs. By employing vector quantization, we compress continuous node embeddings from multiple layers of a Graph Neural Network (GNN) into discrete codes, applicable under both self-supervised and supervised learning paradigms. These node IDs capture high-level abstractions of graph data and offer interpretability that traditional GNN embeddings lack. Extensive experiments on 34 datasets, encompassing node classification, graph classification, link prediction, and attributed graph clustering tasks, demonstrate that the generated node IDs significantly enhance speed and memory efficiency while achieving competitive performance compared to current state-of-the-art methods.

Cited by

• A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning
• Can Mamba Learn In Context with Outliers? A Theoretical Generalization Analysis
• Two Facets of the Same Optimization Coin: Model Degradation and Representation Collapse in Graph Foundation Models
• Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning
• Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey
• Theoretical Learning Performance of Graph Neural Networks: The Impact of Jumping Connections and Layer-wise Sparsification

Related

• Deep Graph Infomax
• Inferential SIR-GN: Scalable Graph Representation Learning
• Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node\n Representation Learning
• Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages
• Graph Neural Networks Powered by Encoder Embedding for Improved Node Learning