2023/03/31 by Yanjie Dong, Luya Wang, Dong, Yanjie +13
Computer Science · Engineering · #Advanced MIMO Systems Optimization #Cooperative Communication and Network Coding #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2303.17885
openalex publication_date 2023/03/31 · openalex created_date 2023/04/06 · openalex updated_date 2026/07/28
A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via orthogonal wireless channels. Since the workers frequently upload local gradients to the server via bandwidth-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. Therefore, a one-shot distributed principle component analysis (PCA) is leveraged to reduce the dimension of uploaded gradients such that the communication bottleneck is relieved. A PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients and the Nesterov's momentum. For the non-convex loss functions, a finite-time analysis is performed to quantify the impacts of system hyper-parameters on the convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is theoretically certified to converge faster than the PCA-WFL algorithm. Besides, the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal the linear speedup with respect to the number of workers over the vanilla gradient descent algorithm. Numerical results are used to demonstrate the improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over the benchmarks.