A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data

2023/09/03 by Reguieg, Hamza, Hanjri, Mohammed El, Kamili, Mohamed El +1 · 4 citations
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Probability (math.PR)

paper · doi:10.48550/arxiv.2309.01275

Abstract

In this paper, we investigate Federated Learning (FL), a paradigm of machine learning that allows for decentralized model training on devices without sharing raw data, there by preserving data privacy. In particular, we compare two strategies within this paradigm: Federated Averaging (FedAvg) and Personalized Federated Averaging (Per-FedAvg), focusing on their performance with Non-Identically and Independently Distributed (Non-IID) data. Our analysis shows that the level of data heterogeneity, modeled using a Dirichlet distribution, significantly affects the performance of both strategies, with Per-FedAvg showing superior robustness in conditions of high heterogeneity. Our results provide insights into the development of more effective and efficient machine learning strategies in a decentralized setting.

Cited by

• Local Performance vs. Out-of-Distribution Generalization: An Empirical Analysis of Personalized Federated Learning in Heterogeneous Data Environments
• Hierarchical Federated Learning for Crop Yield Prediction in Smart Agricultural Production Systems
• Privacy-Preserving Personalization in Education: A Federated Recommender System for Student Performance Prediction
• Centralized vs. Federated Learning for Educational Data Mining: A Comparative Study on Student Performance Prediction with SAEB Microdata

Related

• Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
• Distributionally Robust Federated Learning: An ADMM Algorithm
• Decentralized Federated Averaging via Random Walk
• FedH2L: Federated Learning with Model and Statistical Heterogeneity
• FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity