Mixture of Experts for Node Classification

2024/11/30 by Shixiao Yu, Yiqi Wang, Shi, Yu +8 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Expert finding and Q&A systems #FOS: Computer and information sciences #Social and Information Networks (cs.SI)

paper · pdf · doi:10.48550/arxiv.2412.00418

openalex publication_date 2024/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Nodes in the real-world graphs exhibit diverse patterns in numerous aspects, such as degree and homophily. However, most existent node predictors fail to capture a wide range of node patterns or to make predictions based on distinct node patterns, resulting in unsatisfactory classification performance. In this paper, we reveal that different node predictors are good at handling nodes with specific patterns and only apply one node predictor uniformly could lead to suboptimal result. To mitigate this gap, we propose a mixture of experts framework, MoE-NP, for node classification. Specifically, MoE-NP combines a mixture of node predictors and strategically selects models based on node patterns. Experimental results from a range of real-world datasets demonstrate significant performance improvements from MoE-NP.

Cited by

• ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
• Variational Mixture of Graph Neural Experts for Alzheimer's Disease Recognition across Frequency Bands in EEG Brain Networks

Related

• Node-wise Filtering in Graph Neural Networks: A Mixture of Experts Approach
• Mixture of Weak & Strong Experts on Graphs
• Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling
• Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
• Graph Knowledge Distillation to Mixture of Experts