2021/07/26 by Raz Paul, Paul, Raz, Yuval Friedman +3 · 1 citation
Computer Science · Engineering · #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2107.12452
openalex publication_date 2021/07/26 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
We consider a distributed learning problem in a wireless network, consisting\nof N distributed edge devices and a parameter server (PS). The objective\nfunction is a sum of the edge devices' local loss functions, who aim to train a\nshared model by communicating with the PS over multiple access channels (MAC).\nThis problem has attracted a growing interest in distributed sensing systems,\nand more recently in federated learning, known as over-the-air computation. In\nthis paper, we develop a novel Accelerated Gradient-descent Multiple Access\n(AGMA) algorithm that uses momentum-based gradient signals over noisy fading\nMAC to improve the convergence rate as compared to existing methods.\nFurthermore, AGMA does not require power control or beamforming to cancel the\nfading effect, which simplifies the implementation complexity. We analyze AGMA\ntheoretically, and establish a finite-sample bound of the error for both convex\nand strongly convex loss functions with Lipschitz gradient. For the strongly\nconvex case, we show that AGMA approaches the best-known linear convergence\nrate as the network increases. For the convex case, we show that AGMA\nsignificantly improves the sub-linear convergence rate as compared to existing\nmethods. Finally, we present simulation results using real datasets that\ndemonstrate better performance by AGMA.\n