FedTune: Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective

2021/10/06 by Zhang, Huanle, Zhang, Mi, Liu, Xin +2
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2110.03061

Abstract

Federated learning (FL) hyper-parameters significantly affect the training overheads in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters puts a high burden on FL practitioners since various applications prefer different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements of FL training. FedTune is lightweight and flexible, achieving 8.48%-26.75% improvement for different datasets compared to fixed FL hyper-parameters.

Related

• Federated Learning Hyper-Parameter Tuning from a System Perspective
• Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism
• Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysis
• FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent
• Federated Hypergradient Descent