Private Federated Learning with Autotuned Compression

2023/07/20 by Enayat Ullah, Christopher A. Choquette-Choo, Ullah, Enayat +5 · 1 citation
Computer Science · #Privacy-Preserving Technologies in Data #Cryptography and Data Security #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2307.10999

Abstract

We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem" with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.

Cited by

• Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Models

Related

• Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design
• Compression Boosts Differentially Private Federated Learning
• SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
• Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks
• Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation