2021/03/01 by Xiao Guo, Xiang Li, Guo, Xiao +7 · 1 citation
Computer Science · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data #Random Matrices and Applications #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2103.00704
openalex publication_date 2021/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Eigenspace estimation is fundamental in machine learning and statistics, which has found applications in PCA, dimension reduction, and clustering, among others. The modern machine learning community usually assumes that data come from and belong to different organizations. The low communication power and the possible privacy breaches of data make the computation of eigenspace challenging. To address these challenges, we propose a class of algorithms called \textsfFedPower within the federated learning (FL) framework. \textsfFedPower leverages the well-known power method by alternating multiple local power iterations and a global aggregation step, thus improving communication efficiency. In the aggregation, we propose to weight each local eigenvector matrix with \it Orthogonal Procrustes Transformation (OPT) for better alignment. To ensure strong privacy protection, we add Gaussian noise in each iteration by adopting the notion of differential privacy (DP). We provide convergence bounds for \textsfFedPower that are composed of different interpretable terms corresponding to the effects of Gaussian noise, parallelization, and random sampling of local machines. Additionally, we conduct experiments to demonstrate the effectiveness of our proposed algorithms.