Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

2025/06/11 by Maximilian Egger, Egger, Maximilian, Rawad Bitar +1
Computer Science · #Cryptography and Data Security #Distributed #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2506.09870

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Ensuring resilience to Byzantine clients while maintaining the privacy of the clients' data is a fundamental challenge in federated learning (FL). When the clients' data is homogeneous, suitable countermeasures were studied from an information-theoretic perspective utilizing secure aggregation techniques while ensuring robust aggregation of the clients' gradients. However, the countermeasures used fail when the clients' data is heterogeneous. Suitable pre-processing techniques, such as nearest neighbor mixing, were recently shown to enhance the performance of those countermeasures in the heterogeneous setting. Nevertheless, those pre-processing techniques cannot be applied with the introduced privacy-preserving mechanisms. We propose a multi-stage method encompassing a careful co-design of verifiable secret sharing, secure aggregation, and a tailored symmetric private information retrieval scheme to achieve information-theoretic privacy guarantees and Byzantine resilience under data heterogeneity. We evaluate the effectiveness of our scheme on a variety of attacks and show how it outperforms the previously known techniques. Since the communication overhead of secure aggregation is non-negligible, we investigate the interplay with zero-order estimation methods that reduce the communication cost in state-of-the-art FL tasks and thereby make private aggregation scalable.

Citations

• Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning
• PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning
• Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization
• LoByITFL: Low Communication Secure and Private Federated Learning
• Byzantine-Resilient Secure Aggregation for Federated Learning Without Privacy Compromises
• Secure Aggregation is Not Private Against Membership Inference Attacks
• Private Aggregation in Hierarchical Wireless Federated Learning with Partial and Full Collusion
• Fine-Tuning Language Models with Just Forward Passes
• ByzSecAgg: A Byzantine-Resistant Secure Aggregation Scheme for Federated Learning Based on Coded Computing and Vector Commitment
• Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity
• SwiftAgg+: Achieving Asymptotically Optimal Communication Loads in Secure Aggregation for Federated Learning
• CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning
• LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning
• Approximate Byzantine Fault-Tolerance in Distributed Optimization
• Byzantine-Resilient Non-Convex Stochastic Gradient Descent
• FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
• Learning from History for Byzantine Robust Optimization
• Byzantine-Resilient Secure Federated Learning
• Defending against Backdoors in Federated Learning with Robust Learning\n Rate
• Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Robust Aggregation for Federated Learning
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
• Private Information Retrieval from Coded Storage Systems with Colluding,\n Byzantine, and Unresponsive Servers
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
• Learning from Untrusted Data
• Deep Learning with Limited Numerical Precision
• How to share a secret

Related

• Byzantine-Resilient Secure Aggregation for Federated Learning Without Privacy Compromises
• Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning
• Secure and Private Federated Learning: Achieving Adversarial Resilience through Robust Aggregation
• Byzantine-Resilient Secure Federated Learning
• Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learning