2019/08/21 by Hu, Chenghao, Jiang, Jingyan, Wang, Zhi · 4 citations
#Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Networking and Internet Architecture (cs.NI) #Parallel #and Cluster Computing (cs.DC)
paper · doi:10.48550/arxiv.1908.07782
The emerging concern about data privacy and security has motivated the proposal of federated learning, which allows nodes to only synchronize the locally-trained models instead their own original data. Conventional federated learning architecture, inherited from the parameter server design, relies on highly centralized topologies and the assumption of large nodes-to-server bandwidths. However, in real-world federated learning scenarios the network capacities between nodes are highly uniformly distributed and smaller than that in a datacenter. It is of great challenges for conventional federated learning approaches to efficiently utilize network capacities between nodes. In this paper, we propose a model segment level decentralized federated learning to tackle this problem. In particular, we propose a segmented gossip approach, which not only makes full utilization of node-to-node bandwidth, but also has good training convergence. The experimental results show that even the training time can be highly reduced as compared to centralized federated learning.