Secure Aggregation with Heterogeneous Quantization in Federated Learning

2020/09/30 by Ahmed Roushdy Elkordy, Elkordy, Ahmed Roushdy, Amir Salman Avestimehr +1 · 3 citations
Computer Science · #Cryptography and Data Security #FOS: Computer and information sciences #FOS: Electrical engineering #Information Theory (cs.IT) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2009.14388

openalex publication_date 2020/09/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Secure model aggregation across many users is a key component of federated learning systems. The state-of-the-art protocols for secure model aggregation, which are based on additive masking, require all users to quantize their model updates to the same level of quantization. This severely degrades their performance due to lack of adaptation to available bandwidth at different users. We propose three schemes that allow secure model aggregation while using heterogeneous quantization. This enables the users to adjust their quantization proportional to their available bandwidth, which can provide a substantially better trade-off between the accuracy of training and the communication time. The proposed schemes are based on a grouping strategy by partitioning the network into groups, and partitioning the local model updates of users into segments. Instead of applying aggregation protocol to the entire local model update vector, it is applied on segments with specific coordination between users. We theoretically evaluate the quantization error for our schemes, and also demonstrate how our schemes can be utilized to overcome Byzantine users.

Citations

• How to share a secret
• New directions in cryptography
• Federated Learning: Strategies for Improving Communication Efficiency
• Distributed Mean Estimation with Limited Communication
• Gradient Sparsification for Communication-Efficient Distributed Optimization
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Zeno: Distributed Stochastic Gradient Descent with Suspicion-based\n Fault-tolerance
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning
• ByGARS: Byzantine SGD with Arbitrary Number of Attackers
• The Byzantine Generals Problem

Cited by

• Optimal Key Rates for Decentralized Secure Aggregation with Arbitrary Collusion and Heterogeneous Security Constraints
• On hierarchical secure aggregation against relay and user collusion
• Progressive Size-Adaptive Federated Learning: A Comprehensive Framework for Heterogeneous Multi-Modal Data Systems

Related

• Reconciling Security and Communication Efficiency in Federated Learning
• LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning
• One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning
• Aggregation Service for Federated Learning: An Efficient, Secure, and More Resilient Realization
• Eluding Secure Aggregation in Federated Learning via Model Inconsistency