Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning

2025/06/11 by Haruki Kainuma, Takayuki Nishio, Kainuma, Haruki +1
Computer Science · Engineering · #Advanced Data and IoT Technologies #Age of Information Optimization #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2506.09769

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes Load-aware Tram-FL, an extension of Tram-FL that introduces a training scheduling mechanism to minimize total training time in decentralized federated learning by accounting for both computational and communication loads. The scheduling problem is formulated as a global optimization task, which-though intractable in its original form-is made solvable by decomposing it into node-wise subproblems. To promote balanced data utilization under non-IID distributions, a variance constraint is introduced, while the overall training latency, including both computation and communication costs, is minimized through the objective function. Simulation results on MNIST and CIFAR-10 demonstrate that Load-aware Tram-FL significantly reduces training time and accelerates convergence compared to baseline methods.

Citations

• Tram-FL: Routing-based Model Training for Decentralized Federated Learning
• Model-Based Derivative-Free Optimization Methods and Software
• Gossip training for deep learning
• How to scale distributed deep learning?
• Deep Residual Learning for Image Recognition
• Gradient-based learning applied to document recognition

Related

• Load Balancing in Federated Learning
• Tram-FL: Routing-based Model Training for Decentralized Federated Learning
• Scheduling Algorithms for Federated Learning with Minimal Energy Consumption
• RIFLES: Resource-effIcient Federated LEarning via Scheduling
• Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach