Connecting Federated ADMM to Bayes

2025/01/28 by Siddharth Swaroop, Mohammad Emtiyaz Khan, Swaroop, Siddharth +3 · 1 citation
Computer Science · #Advanced Database Systems and Queries #Artificial Intelligence (cs.AI) #Distributed systems and fault tolerance #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2501.17325

openalex publication_date 2025/01/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We provide new connections between two distinct federated learning approaches based on (i) ADMM and (ii) Variational Bayes (VB), and propose new variants by combining their complementary strengths. Specifically, we show that the dual variables in ADMM naturally emerge through the 'site' parameters used in VB with isotropic Gaussian covariances. Using this, we derive two versions of ADMM from VB that use flexible covariances and functional regularisation, respectively. Through numerical experiments, we validate the improvements obtained in performance. The work shows connection between two fields that are believed to be fundamentally different and combines them to improve federated learning.

Cited by

• Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning

Related

• Federated ADMM from Bayesian Duality
• Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework
• FedAPM: Federated Learning via ADMM with Partial Model Personalization
• Partitioned Variational Inference: A Framework for Probabilistic Federated Learning
• Federated Variational Inference: Towards Improved Personalization and Generalization