Federated Learning with Feedback Alignment

2025/12/14 by Incheol Baek, Baek, Incheol, Hyungbin Kim +4
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Data Stream Mining Techniques #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2512.12762

openalex publication_date 2025/12/14 · openalex created_date 2025/12/17 · openalex updated_date 2026/07/28

Abstract

Federated Learning (FL) enables collaborative training across multiple clients while preserving data privacy, yet it struggles with data heterogeneity, where clients' data are not distributed independently and identically (non-IID). This causes local drift, hindering global model convergence. To address this, we introduce Federated Learning with Feedback Alignment (FLFA), a novel framework that integrates feedback alignment into FL. FLFA uses the global model's weights as a shared feedback matrix during local training's backward pass, aligning local updates with the global model efficiently. This approach mitigates local drift with minimal additional computational cost and no extra communication overhead. Our theoretical analysis supports FLFA's design by showing how it alleviates local drift and demonstrates robust convergence for both local and global models. Empirical evaluations, including accuracy comparisons and measurements of local drift, further illustrate that FLFA can enhance other FL methods demonstrating its effectiveness.

Citations

• FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning
• Federated Learning with Label Distribution Skew via Logits Calibration
• Communication-Efficient Federated Learning with Accelerated Client Gradient
• Federated Learning Based on Dynamic Regularization
• Model-Contrastive Federated Learning
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Federated Learning with Matched Averaging
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• On the Convergence of FedAvg on Non-IID Data
• Robust and Communication-Efficient Federated Learning from Non-IID Data
• Agnostic Federated Learning
• Federated Optimization in Heterogeneous Networks
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Direct Feedback Alignment Provides Learning in Deep Neural Networks
• Deep Residual Learning for Image Recognition
• ImageNet Large Scale Visual Recognition Challenge

Related

• Rethinking Client Drift in Federated Learning: A Logit Perspective
• Global Update Guided Federated Learning
• FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning
• FedShuffle: Recipes for Better Use of Local Work in Federated Learning
• FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction