Eavesdrop the Composition Proportion of Training Labels in Federated Learning

2019/10/14 by Lixu Wang, Wang, Lixu, Shichao Xu +5 · 35 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Artificial intelligence #Business #Composition (language) #Computer science #Cryptography and Data Security #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Federated learning #Geography #Linguistics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multiagent Systems (cs.MA) #Privacy-Preserving Technologies in Data #Training (meteorology) #cs.CR #cs.LG #cs.MA #stat.ML

paper · pdf · doi:10.48550/arxiv.1910.06044

published in arXiv (Cornell University) (Cornell University) · 15 pages, 10 figures, security conference

openalex publication_date 2019/10/14 · arxiv created 2019/10/27 · arxiv updated 2019/10/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.

Citations

• Gradient-based learning applied to document recognition
• Federated Learning: Strategies for Improving Communication Efficiency
• Differentially Private Federated Learning: A Client Level Perspective
• How To Backdoor Federated Learning
• Mitigating Sybils in Federated Learning Poisoning
• Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning

Cited by

• Adversarial Robustness in Distributed Quantum Machine Learning
• EC-LDA : Label Distribution Inference Attack against Federated Graph Learning with Embedding Compression
• 2-Step Agent: A Framework for the Interaction of a Decision Maker with AI Decision Support

Related

• Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups
• Poison to Detect: Detection of Targeted Overfitting in Federated Learning
• Source Inference Attacks in Federated Learning
• A Survey on Federated Learning Poisoning Attacks and Defenses
• Poisoning Attacks and Defenses in Federated Learning: A Survey