One-shot Robust Federated Learning of Independent Component Analysis

2025/05/26 by Jin, Dian, Xin Bing, Yuqian Zhang +2
Computer Science · #Biometric Identification and Security #Blind Source Separation Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Wireless Signal Modulation Classification

paper · pdf · doi:10.48550/arxiv.2505.20532

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper investigates a general robust one-shot aggregation framework for distributed and federated Independent Component Analysis (ICA) problem. We propose a geometric median-based aggregation algorithm that leverages k-means clustering to resolve the permutation ambiguity in local client estimations. Our method first performs k-means to partition client-provided estimators into clusters and then aggregates estimators within each cluster using the geometric median. This approach provably remains effective even in highly heterogeneous scenarios where at most half of the clients can observe only a minimal number of samples. The key theoretical contribution lies in the combined analysis of the geometric median's error bound-aided by sample quantiles-and the maximum misclustering rates of the aforementioned solution of k-means. The effectiveness of the proposed approach is further supported by simulation studies conducted under various heterogeneous settings.

Citations

• Task Arithmetic Through The Lens Of One-Shot Federated Learning
• FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion
• One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversity
• FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
• Optimal vintage factor analysis with deflation varimax
• FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation
• Large Dimensional Independent Component Analysis: Statistical Optimality and Computational Tractability
• A penalized criterion for selecting the number of clusters for K-medians
• Global Convergence of Federated Learning for Mixed Regression
• Private Non-Convex Federated Learning Without a Trusted Server
• Faster Non-Convex Federated Learning via Global and Local Momentum
• From Symmetry to Geometry: Tractable Nonconvex Problems
• Robust Aggregation for Federated Learning
• Advances and Open Problems in Federated Learning
• On the Convergence of Local Descent Methods in Federated Learning
• Tighter Theory for Local SGD on Identical and Heterogeneous Data
• On the Convergence of FedAvg on Non-IID Data
• Robust Federated Learning in a Heterogeneous Environment
• Federated Optimization in Heterogeneous Networks
• Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Polynomial-time Tensor Decompositions with Sum-of-Squares
• Analyzing Tensor Power Method Dynamics in Overcomplete Regime
• Distributed Robust Learning
• Improved Spectral-Norm Bounds for Clustering
• Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm
• Clustering with Spectral Norm and the k-means Algorithm
• Approximation algorithms for metric facility location and k -Median problems using the primal-dual schema and Lagrangian relaxation
• Independent component analysis: algorithms and applications
• Optimum Locations of Switching Centers and the Absolute Centers and Medians of a Graph
• A (2+ε)-Approximation Algorithm for Metric k-Median
• Regression Quantiles

Related

• Robust Aggregation for Federated Learning
• FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation
• Jigsaw Game: Federated Clustering
• Robust and Efficient Aggregation for Distributed Learning
• Federated Learning Aggregation: New Robust Algorithms with Guarantees