FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning

2025/10/23 by Yang Zhi-qin, Yonggang Zhang, Yang, Zhiqin +9 · 1 citation
Computer Science · Decision Sciences · #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2510.20250

openalex publication_date 2025/10/23 · openalex created_date 2025/10/25 · openalex updated_date 2026/07/28

Abstract

Federated Learning (FL) confronts a significant challenge known as data heterogeneity, which impairs model performance and convergence. Existing methods have made notable progress in addressing this issue. However, improving performance in certain heterogeneity scenarios remains an overlooked question: How robust are these methods to deploy under diverse heterogeneity scenarios? To answer this, we conduct comprehensive evaluations across varied heterogeneity scenarios, showing that most existing methods exhibit limited robustness. Meanwhile, insights from these experiments highlight that sharing statistical information can mitigate heterogeneity by enabling clients to update with a global perspective. Motivated by this, we propose FedGPS (Federated Goal-Path Synergy), a novel framework that seamlessly integrates statistical distribution and gradient information from others. Specifically, FedGPS statically modifies each client's learning objective to implicitly model the global data distribution using surrogate information, while dynamically adjusting local update directions with gradient information from other clients at each round. Extensive experiments show that FedGPS outperforms state-of-the-art methods across diverse heterogeneity scenarios, validating its effectiveness and robustness. The code is available at: https://github.com/CUHK-AIM-Group/FedGPS.

Citations

• Does Training with Synthetic Data Truly Protect Privacy?
• Unlocking the Potential of Model Calibration in Federated Learning
• Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning
• Statistical Multicriteria Benchmarking via the GSD-Front
• Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization
• FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfer
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity
• Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains
• Causal Multi-Label Feature Selection in Federated Setting
• Robust Training of Federated Models with Extremely Label Deficiency
• Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Study
• PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning
• Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark
• Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition
• Gradual Domain Adaptation: Theory and Algorithms
• Locally Adaptive Federated Learning
• Adaptive Federated Learning with Auto-Tuned Clients
• Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization
• FedDisco: Federated Learning with Discrepancy-Aware Collaboration
• FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment
• Revisiting Weighted Aggregation in Federated Learning with Neural Networks
• FedExP: Speeding Up Federated Averaging via Extrapolation
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning
• Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
• Generalized Federated Learning via Sharpness Aware Minimization
• Improving Generalization in Federated Learning by Seeking Flat Minima
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
• Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
• Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning
• FedProto: Federated Prototype Learning across Heterogeneous Clients
• Model-Contrastive Federated Learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Optimal Client Sampling for Federated Learning
• Sharpness-Aware Minimization for Efficiently Improving Generalization
• Improving Generative Adversarial Networks with Local Coordinate Coding
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Adaptive Federated Optimization
• Advances and Open Problems in Federated Learning
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Federated Learning: Challenges, Methods, and Future Directions
• Federated Optimization in Heterogeneous Networks
• How To Backdoor Federated Learning

Cited by

• FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning

Related

• FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learning
• Gradient Masked Averaging for Federated Learning
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity
• Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks
• FedImpro: Measuring and Improving Client Update in Federated Learning