Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent

2020/09/11 by Rahif Kassab, Osvaldo Simeone, Kassab, Rahif +1 · 4 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Electrical engineering #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data #Signal Processing (eess.SP) #Stochastic Gradient Optimization Techniques #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2009.06419

openalex publication_date 2020/09/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning. DSVGD maintains a number of non-random and interacting particles at a central server to represent the current iterate of the model global posterior. The particles are iteratively downloaded and updated by one of the agents with the end goal of minimizing the global free energy. By varying the number of particles, DSVGD enables a flexible trade-off between per-iteration communication load and number of communication rounds. DSVGD is shown to compare favorably to benchmark frequentist and Bayesian federated learning strategies, also scheduling a single device per iteration, in terms of accuracy and scalability with respect to the number of agents, while also providing well-calibrated, and hence trustworthy, predictions.

Cited by

• Adaptive Kernel Selection for Stein Variational Gradient Descent
• Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning
• FedUP: Efficient Pruning-based Federated Unlearning for Model Poisoning Attacks
• Federated Learning in Active STARS-Aided Uplink Networks

Related

• Forget-SVGD: Particle-Based Bayesian Federated Unlearning
• Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agents
• Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
• FedHB: Hierarchical Bayesian Federated Learning
• Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework