Differentially Private Federated Learning: A Client Level Perspective

2017/12/20 by R. Geyer, Robin C. Geyer, Tassilo Klein +4 · 162 citations
Computer Science · Mathematics · #Cryptography and Data Security #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #cs.CR #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1712.07557

NIPS 2017 Workshop: Machine Learning on the Phone and other Consumer Devices

arxiv created 2018/03/01 · arxiv updated 2018/03/02

Abstract

Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance.

Cited by

• Multi-Agent Privacy Game in Federated Learning: A Unified Mean-Field View
• First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions
• Empirical Evaluation of Structured Synthetic Data Privacy Metrics: Novel experimental framework
• Evaluating Adversarial Attacks on Federated Learning for Temperature Forecasting
• Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain
• DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm
• Private Multi-Task Learning: Formulation and Applications to Federated Learning
• PowerScale: Energy-Efficient Geo-Distributed Model Training with Federated Datacenter Power
• Decentralized Differentially Private Segmentation with PATE
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communication
• SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Mitigating Sybils in Federated Learning Poisoning
• Federated Intrusion Detection for IoT with Heterogeneous Cohort Privacy
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• Asynchronous Federated Learning with Differential Privacy for Edge Intelligence
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Byzantine-Resilient Secure Federated Learning
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Local Differential Privacy and Its Applications: A Comprehensive Survey
• Understanding the Tradeoffs in Client-side Privacy for Downstream Speech Tasks
• Federated Learning With Differential Privacy: Algorithms and Performance Analysis
• Enhancing the Privacy of Federated Learning with Sketching
• FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Differential Privacy as a Perk: Federated Learning over Multiple-Access Fading Channels with a Multi-Antenna Base Station
• Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
• Separation of Powers in Federated Learning
• WAFFLE: Watermarking in Federated Learning
• A survey on federated learning
• Local Differential Privacy for Federated Learning with Fixed Memory Usage and Per-Client Privacy
• FedGTEA: Federated Class-Incremental Learning with Gaussian Task Embedding and Alignment
• LEAF: A Benchmark for Federated Settings
• Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges
• Evaluation of Differential Privacy Mechanisms on Federated Learning
• Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• DP-HYPE: Distributed Differentially Private Hyperparameter Search
• Non-Linear Trajectory Modeling for Multi-Step Gradient Inversion Attacks in Federated Learning
• Gradient-Leakage Resilient Federated Learning
• Private Federated Learning Without a Trusted Server: Optimal Algorithms\n for Convex Losses
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times
• Security and Privacy Challenges of Large Language Models: A Survey
• Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
• Topology-aware Differential Privacy for Decentralized Image Classification
• Differential Privacy for Euclidean Jordan Algebra with Applications to Private Symmetric Cone Programming
• Federated Learning with Differential Privacy: Algorithms and Performance Analysis
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Differentially private federated learning for localized control of infectious disease dynamics
• PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy
• Free-riders in Federated Learning: Attacks and Defenses
• Privacy-preserving Federated Bayesian Learning of a Generative Model for Imbalanced Classification of Clinical Data
• From Federated Learning to Federated Neural Architecture Search: A Survey
• Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
• FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection
• Decentralized Wireless Federated Learning with Differential Privacy
• PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• Trustworthy Artificial Intelligence: A Review
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization
• On the Convergence of FedAvg on Non-IID Data
• Dynamic Attention-based Communication-Efficient Federated Learning
• Rethinking Layer-wise Gaussian Noise Injection: Bridging Implicit Objectives and Privacy Budget Allocation
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Gaming and Cooperation in Federated Learning: What Can Happen and How to Monitor It
• LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization
• Enhancing Model Privacy in Federated Learning with Random Masking and Quantization
• Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective
• Membership Inference Attacks and Defenses in Federated Learning: A Survey
• Critical Learning Periods in Federated Learning
• FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
• Edge Intelligence: Architectures, Challenges, and Applications
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• SelectiveShield: Lightweight Hybrid Defense Against Gradient Leakage in Federated Learning
• Differentially Private Meta-Learning
• FDNAS: Improving Data Privacy and Model Diversity in AutoML
• Cyst-X: A Federated AI System Outperforms Clinical Guidelines to Detect Pancreatic Cancer Precursors and Reduce Unnecessary Surgery
• Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection
• Abnormal Client Behavior Detection in Federated Learning
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy
• Differentially-Private Federated Linear Bandits
• Curie: Policy-based Secure Data Exchange
• An Exploratory Analysis on Users' Contributions in Federated Learning
• Federated Model Distillation with Noise-Free Differential Privacy
• Preventing Adversarial Use of Datasets through Fair Core-Set Construction
• FedGA: A Fair Federated Learning Framework Based on the Gini Coefficient
• Fair and autonomous sharing of federate learning models in mobile Internet of Things
• Advances and Open Problems in Federated Learning
• FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning
• Privacy-Preserving Collaborative Deep Learning with Unreliable Participants
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• Convergence of Agnostic Federated Averaging
• Learn distributed GAN with Temporary Discriminators
• Federated Learning from Small Datasets
• Learning Private Representations through Entropy-based Adversarial Training
• DRAGD: A Federated Unlearning Data Reconstruction Attack Based on Gradient Differences
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• Anonymizing Data for Privacy-Preserving Federated Learning
• One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning
• A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks
• Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
• Asymptotically Optimal Secure Aggregation for Wireless Federated Learning with Multiple Servers
• GFL: A Decentralized Federated Learning Framework Based On Blockchain
• Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning
• MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
• FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
• Decoding Federated Learning: The FedNAM+ Conformal Revolution
• Eavesdrop the Composition Proportion of Training Labels in Federated Learning
• Differentially Private Learning Needs Better Features (or Much More Data)
• Auditing Data Provenance in Text-Generation Models
• Federated Multi-Armed Bandits
• Federated Learning in Adversarial Settings
• Similarity Weighted Aggregation with Global Differential Privacy for Federated Brain Lesion Segmentation
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• Pronto: Federated Task Scheduling
• Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
• Evaluating Differentially Private Machine Learning in Practice
• Combining Federated and Active Learning for Communication-efficient Distributed Failure Prediction in Aeronautics
• Voting-based Approaches For Differentially Private Federated Learning
• Towards One-shot Federated Learning: Advances, Challenges, and Future Directions
• A Federated Learning Approach for Mobile Packet Classification
• Multimodal Federated Learning: A Survey through the Lens of Different FL Paradigms
• Private Federated Learning with Domain Adaptation
• Applications of federated learning in smart cities: recent advances, taxonomy, and open challenges
• LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environments
• An Overview of Privacy in Machine Learning
• D2P-Fed: Differentially Private Federated Learning With Efficient Communication
• Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning
• Source Inference Attacks in Federated Learning
• Enhancing Federated Survival Analysis through Peer-Driven Client Reputation in Healthcare
• Federated Learning for Healthcare Informatics
• A Theoretical Perspective on Differentially Private Federated Multi-task Learning
• Robust Federated Learning: The Case of Affine Distribution Shifts
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning
• Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning
• An Empirical Study on the Intrinsic Privacy of SGD
• Confederated Machine Learning on Horizontally and Vertically Separated Medical Data for Large-Scale Health System Intelligence
• Empirical Analysis of Asynchronous Federated Learning on Heterogeneous Devices: Efficiency, Fairness, and Privacy Trade-offs
• RiM: Record, Improve and Maintain Physical Well-being using Federated Learning
• Efficient Full-Stack Private Federated Deep Learning with Post-Quantum Security
• Federated f-Differential Privacy
• PQS-BFL: A Post-Quantum Secure Blockchain-based Federated Learning Framework
• Federated Learning: Opportunities and Challenges
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Federated Extra-Trees with Privacy Preserving
• Federated Learning: A Survey on Privacy-Preserving Collaborative Intelligence
• Federated Unbiased Learning to Rank
• Federated Learning System without Model Sharing through Integration of Dimensional Reduced Data Representations
• Federated Forest
• Towards Communication-Efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• Federated Heavy Hitters Discovery with Differential Privacy
• Differential Privacy Has Disparate Impact on Model Accuracy
• Trends and Advancements in Deep Neural Network Communication
• A Secure Federated Learning Framework for 5G Networks
• Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages
• Fast-adapting and Privacy-preserving Federated Recommender System

Related

• Optimal Strategies for Federated Learning Maintaining Client Privacy
• Differential Privacy in Federated Learning: Mitigating Inference Attacks with Randomized Response
• Federated Learning with Differential Privacy
• Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacy
• Federated Learning in Adversarial Settings