2020/12/10 by Canh T. Dinh, Nguyen H. Tran, Dinh, Canh T. +11 · 1 citation
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2012.05625
openalex publication_date 2020/12/10 · openalex created_date 2022/07/19 · openalex updated_date 2026/07/28
There is growing interest in applying distributed machine learning to edge\ncomputing, forming federated edge learning. Federated edge learning faces\nnon-i.i.d. and heterogeneous data, and the communication between edge workers,\npossibly through distant locations and with unstable wireless networks, is more\ncostly than their local computational overhead. In this work, we propose DONE,\na distributed approximate Newton-type algorithm with fast convergence rate for\ncommunication-efficient federated edge learning. First, with strongly convex\nand smooth loss functions, DONE approximates the Newton direction in a\ndistributed manner using the classical Richardson iteration on each edge\nworker. Second, we prove that DONE has linear-quadratic convergence and analyze\nits communication complexities. Finally, the experimental results with\nnon-i.i.d. and heterogeneous data show that DONE attains a comparable\nperformance to the Newton's method. Notably, DONE requires fewer communication\niterations compared to distributed gradient descent and outperforms DANE and\nFEDL, state-of-the-art approaches, in the case of non-quadratic loss functions.\n