PracMHBench: Re-evaluating Model-Heterogeneous Federated Learning Based on Practical Edge Device Constraints

2025/09/04 by Yuanchun Guo, Guo, Yuanchun, Bingyan Liu +5
Computer Science · #Big Data and Digital Economy #Distributed #FOS: Computer and information sciences #IoT and Edge/Fog Computing #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2509.08750

openalex publication_date 2025/09/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Federating heterogeneous models on edge devices with diverse resource constraints has been a notable trend in recent years. Compared to traditional federated learning (FL) that assumes an identical model architecture to cooperate, model-heterogeneous FL is more practical and flexible since the model can be customized to satisfy the deployment requirement. Unfortunately, no prior work ever dives into the existing model-heterogeneous FL algorithms under the practical edge device constraints and provides quantitative analysis on various data scenarios and metrics, which motivates us to rethink and re-evaluate this paradigm. In our work, we construct the first system platform PracMHBench to evaluate model-heterogeneous FL on practical constraints of edge devices, where diverse model heterogeneity algorithms are classified and tested on multiple data tasks and metrics. Based on the platform, we perform extensive experiments on these algorithms under the different edge constraints to observe their applicability and the corresponding heterogeneity pattern.

Citations

• Memory-adaptive Depth-wise Heterogeneous Federated Learning
• FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction
• Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning
• FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server
• FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• FedProto: Federated Prototype Learning across Heterogeneous Clients
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• Federated Learning on Non-IID Data Silos: An Experimental Study
• FedEval: A Holistic Evaluation Framework for Federated Learning
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Towards Federated Learning at Scale: System Design
• LEAF: A Benchmark for Federated Settings
• Attention Is All You Need
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Deep Residual Learning for Image Recognition
• Character-level Convolutional Networks for Text Classification
• Going Deeper with Convolutions

Related

• FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing
• FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices
• ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issues
• Model Pruning Enables Efficient Federated Learning on Edge Devices
• Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout