Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement

2025/05/19 by Yinlin Zhu, Zhu, Yinlin, Xunkai Li +8 · 3 citations
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Databases (cs.DB) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Social and Information Networks (cs.SI)

paper · pdf · doi:10.48550/arxiv.2505.12684

openalex publication_date 2025/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advances in graph machine learning have shifted to data-centric paradigms, driven by two emerging fields: (1) Federated graph learning (FGL) enables multi-client collaboration but faces challenges from data and task heterogeneity, limiting its practicality; (2) Graph foundation models (GFM) offer strong domain generalization but are usually trained on single machines, missing out on cross-silo data and resources. These paradigms are complementary, and their integration brings notable benefits. Motivated by this, we propose FedGFM, a novel decentralized GFM training paradigm. However, a key challenge is knowledge entanglement, where multi-domain knowledge merges into indistinguishable representations, hindering downstream adaptation. To address this, we present FedGFM+, an enhanced framework with two core modules to reduce knowledge entanglement: (1) AncDAI: A global anchor-based domain-aware initialization strategy. Before pre-training, each client encodes its local graph into domain-specific prototypes that serve as semantic anchors. Synthetic embeddings around these anchors initialize the global model. We theoretically prove these prototypes are distinguishable across domains, providing a strong inductive bias to disentangle domain-specific knowledge. (2) AdaDPP: A local adaptive domain-sensitive prompt pool. Each client learns a lightweight graph prompt capturing domain semantics during pre-training. During fine-tuning, prompts from all clients form a pool from which the GFM selects relevant prompts to augment target graph attributes, improving downstream adaptation. FedGFM+ is evaluated on 8 diverse benchmarks across multiple domains and tasks, outperforming 20 baselines from supervised learning, FGL, and federated GFM variants.

Citations

• LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models
• Rethinking Tokenized Graph Transformers for Node Classification
• SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation
• RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry
• Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment
• GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation
• UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs
• Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees
• GFT: Graph Foundation Model with Transferable Tree Vocabulary
• RAGraph: A General Retrieval-Augmented Graph Learning Framework
• LangGFM: A Large Language Model Alone Can be a Powerful Graph Foundation Model
• Learning Graph Quantized Tokenizers
• GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs
• AnyGraph: Graph Foundation Model in the Wild
• Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models
• FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning
• OpenGraph: Towards Open Graph Foundation Models
• UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs
• AdaFGL: A New Paradigm for Federated Node Classification with Topology Heterogeneity
• Deep Efficient Private Neighbor Generation for Subgraph Federated Learning
• One for All: Towards Training One Graph Model for All Classification Tasks
• FedGTA: Topology-aware Averaging for Federated Graph Learning
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Privacy-Preserving Graph Machine Learning from Data to Computation: A Survey
• GPT-4 Technical Report
• Federated Learning on Non-IID Graphs via Structural Knowledge Sharing
• FedEgo: Privacy-preserving Personalized Federated Graph Learning with Ego-graphs
• Personalized Subgraph Federated Learning
• FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning
• PaLM: Scaling Language Modeling with Pathways
• Subgraph Federated Learning with Missing Neighbor Generation
• Federated Graph Classification over Non-IID Graphs
• Federated Graph Learning -- A Position Paper
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Model-Contrastive Federated Learning
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
• Language Models are Few-Shot Learners
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Optuna: A Next-generation Hyperparameter Optimization Framework
• Graph Neural Networks: A Review of Methods and Applications
• Graph neural networks: A review of methods and applications
• How Powerful are Graph Neural Networks?
• Graph Attention Networks
• Convolutional 2D Knowledge Graph Embeddings
• Inductive Representation Learning on Large Graphs
• MoleculeNet: A Benchmark for Molecular Machine Learning
• Semi-Supervised Classification with Graph Convolutional Networks
• Revisiting Semi-Supervised Learning with Graph Embeddings
• Spectral Networks and Locally Connected Networks on Graphs
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• OpenFGL: A Comprehensive Benchmark for Federated Graph Learning
• A Comprehensive Survey on Graph Neural Networks

Cited by

• FedBook: A Unified Federated Graph Foundation Codebook with Intra-domain and Inter-domain Knowledge Modeling
• A Comprehensive Data-centric Overview of Federated Graph Learning
• MMFGU: Multimodal Federated Graph Unlearning

Related

• FedBook: A Unified Federated Graph Foundation Codebook with Intra-domain and Inter-domain Knowledge Modeling
• Knowledge-Driven Federated Graph Learning on Model Heterogeneity
• Federated Prototype Graph Learning
• Federated Graph Learning with Graphless Clients
• Federated Learning on Non-IID Graphs via Structural Knowledge Sharing