Row-wise Fusion Regularization: An Interpretable Personalized Federated Learning Framework in Large-Scale Scenarios

2025/10/16 by Runlin Zhou, Letian Li, Zhou, Runlin +3
Computer Science · #Bridging (networking) #Convergence (economics) #FOS: Computer and information sciences #Federated learning #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mobile Crowdsensing and Crowdsourcing #Multivariate statistics #Oracle #Position (finance) #Position error #Privacy-Preserving Technologies in Data #Random oracle #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.2510.14413

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/10/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/08/08

Abstract

We study personalized federated learning for multivariate responses where client models are heterogeneous yet share variable-level structure. Existing entry-wise penalties ignore cross-response dependence, while matrix-wise fusion over-couples clients. We propose a Sparse Row-wise Fusion (SROF) regularizer that clusters row vectors across clients and induces within-row sparsity, and we develop RowFed, a communication-efficient federated algorithm that embeds SROF into a linearized ADMM framework with privacy-preserving partial participation. Theoretically, we establish an oracle property for SROF-achieving correct variable-level group recovery with asymptotic normality-and prove convergence of RowFed to a stationary solution. Under random client participation, the iterate gap contracts at a rate that improves with participation probability. Empirically, simulations in heterogeneous regimes show that RowFed consistently lowers estimation and prediction error and strengthens variable-level cluster recovery over NonFed, FedAvg, and a personalized matrix-fusion baseline. A real-data study further corroborates these gains while preserving interpretability. Together, our results position row-wise fusion as an effective and transparent paradigm for large-scale personalized federated multivariate learning, bridging the gap between entry-wise and matrix-wise formulations.

Citations

• Personalized Cross-Silo Federated Learning on Non-IID Data
• An Efficient Framework for Clustered Federated Learning
• Federated Learning with Personalization Layers
• Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints
• SOFAR: large-scale association network learning
• A concave pairwise fusion approach to subgroup analysis
• Nearly unbiased variable selection under minimax concave penalty
• Non-Concave Penalized Likelihood with NP-Dimensionality

Related

• Robustness and Personalization in Federated Learning: A Unified Approach via Regularization
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift
• GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalization
• Federated Matrix Factorization: Algorithm Design and Application to Data Clustering
• On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness