Non-Homophilic Graph Pre-Training and Prompt Learning

2024/08/22 by Xingtong Yu, Jie Zhang, Yu, Xingtong +5 · 10 citations
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2408.12594

openalex publication_date 2024/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Graphs are ubiquitous for modeling complex relationships between objects across various fields. Graph neural networks (GNNs) have become a mainstream technique for graph-based applications, but their performance heavily relies on abundant labeled data. To reduce labeling requirement, pre-training and prompt learning has become a popular alternative. However, most existing prompt methods do not differentiate homophilic and heterophilic characteristics of real-world graphs. In particular, many real-world graphs are non-homophilic, not strictly or uniformly homophilic with mixing homophilic and heterophilic patterns, exhibiting varying non-homophilic characteristics across graphs and nodes. In this paper, we propose ProNoG, a novel pre-training and prompt learning framework for such non-homophilic graphs. First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks. Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks. Finally, we thoroughly evaluate and analyze ProNoG through extensive experiments on ten public datasets.

Cited by

• Task-Aware Retrieval Augmentation for Dynamic Recommendation
• GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning
• GraphTOP: Graph Topology-Oriented Prompting for Graph Neural Networks
• Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
• Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
• Event-Aware Prompt Learning for Dynamic Graphs
• Higher-order interactions of multi-layer prompt
• MolGA: Molecular Graph Adaptation with Pre-trained 2D Graph Encoder
• One Prompt Fits All: Universal Graph Adaptation for Pretrained Models
• Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily

Related

• New Benchmarks for Learning on Non-Homophilous Graphs
• PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks
• Graph Neural Networks for Graphs with Heterophily: A Survey
• Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods
• Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks