Personalized Federated Learning via Variational Bayesian Inference

2022/06/16 by Xu Zhang, Zhang, Xu, Yinchuan Li +7 · 8 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.2206.07977

openalex publication_date 2022/06/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.

Cited by

• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching
• Bayesian Coreset Optimization for Personalized Federated Learning
• Federated Learning via Meta-Variational Dropout
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks
• Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning
• Towards Instance-wise Personalized Federated Learning via Semi-Implicit Bayesian Prompt Tuning
• FIVA: Federated Inverse Variance Averaging for Universal CT Segmentation with Uncertainty Estimation
• High-Performance Statistical Computing (HPSC): Challenges, Opportunities, and Future Directions

Related

• Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering
• Confidence-aware Personalized Federated Learning via Variational Expectation Maximization
• Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune
• Personalized Federated Learning with Hidden Information on Personalized Prior
• Federated Variational Inference: Towards Improved Personalization and Generalization