Hierarchical Federated ADMM

2024/09/27 by Seyed Mohammad Azimi‐Abarghouyi, Nicola Bastianello, Azimi-Abarghouyi, Seyed Mohammad +5
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Database Systems and Queries #Algorithms and Data Compression #Artificial Intelligence (cs.AI) #DNA and Biological Computing #Distributed #FOS: Computer and information sciences #FOS: Electrical engineering #Information Theory (cs.IT) #Machine Learning (cs.LG) #Parallel #Systems and Control (eess.SY) #and Cluster Computing (cs.DC) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2409.18796

openalex publication_date 2024/09/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we depart from the widely-used gradient descent-based hierarchical federated learning (FL) algorithms to develop a novel hierarchical FL framework based on the alternating direction method of multipliers (ADMM). Within this framework, we propose two novel FL algorithms, which both use ADMM in the top layer: one that employs ADMM in the lower layer and another that uses the conventional gradient descent-based approach. The proposed framework enhances privacy, and experiments demonstrate the superiority of the proposed algorithms compared to the conventional algorithms in terms of learning convergence and accuracy. Additionally, gradient descent on the lower layer performs well even if the number of local steps is very limited, while ADMM on both layers lead to better performance otherwise.

Related

• Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learning
• Federated Learning via Inexact ADMM
• Communication-Efficient ADMM-based Federated Learning
• FedAPM: Federated Learning via ADMM with Partial Model Personalization
• Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework