Inverting Gradients -- How easy is it to break privacy in federated learning?

2020/03/31 by Jonas Geiping, Geiping, Jonas, Hartmut Bauermeister +5 · 121 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #cs.CR #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2003.14053

23 pages, 20 figures. The first three authors contributed equally

openalex publication_date 2020/03/31 · arxiv created 2020/09/11 · arxiv updated 2020/09/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The idea of federated learning is to collaboratively train a neural network on a server. Each user receives the current weights of the network and in turns sends parameter updates (gradients) based on local data. This protocol has been designed not only to train neural networks data-efficiently, but also to provide privacy benefits for users, as their input data remains on device and only parameter gradients are shared. But how secure is sharing parameter gradients? Previous attacks have provided a false sense of security, by succeeding only in contrived settings - even for a single image. However, by exploiting a magnitude-invariant loss along with optimization strategies based on adversarial attacks, we show that is is actually possible to faithfully reconstruct images at high resolution from the knowledge of their parameter gradients, and demonstrate that such a break of privacy is possible even for trained deep networks. We analyze the effects of architecture as well as parameters on the difficulty of reconstructing an input image and prove that any input to a fully connected layer can be reconstructed analytically independent of the remaining architecture. Finally we discuss settings encountered in practice and show that even averaging gradients over several iterations or several images does not protect the user's privacy in federated learning applications in computer vision.

Cited by

• Multi-Agent Privacy Game in Federated Learning: A Unified Mean-Field View
• CHRONOS: A Hardware‐Assisted Phase‐Decoupled Framework for Secure Federated Learning in IoT
• zkFL-Health: Blockchain-Enabled Zero-Knowledge Federated Learning for Medical AI Privacy
• FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative
• FedVideoMAE: Efficient Privacy-Preserving Federated Video Moderation
• Optimal Key Rates for Decentralized Secure Aggregation with Arbitrary Collusion and Heterogeneous Security Constraints
• From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learning
• LCMem: A Universal Model for Robust Image Memorization Detection
• DP-EMAR: A Differentially Private Framework for Autonomous Model Weight Repair in Federated IoT Systems
• Fully Decentralized Certified Unlearning
• Differentially Private and Federated Structure Learning in Bayesian Networks
• Teleportation-Based Defenses for Privacy in Approximate Machine Unlearning
• One-Shot Secure Aggregation: A Hybrid Cryptographic Protocol for Private Federated Learning in IoT
• Quantifying the Privacy-Utility Trade-off in GPS-based Daily Stress Recognition using Semantic Features
• Federated Learning Survey: A Multi-Level Taxonomy of Aggregation Techniques, Experimental Insights, and Future Frontiers
• Privacy in Federated Learning with Spiking Neural Networks
• Privacy-Preserving Federated Vision Transformer Learning Leveraging Lightweight Homomorphic Encryption in Medical AI
• Readout-Side Bypass for Residual Hybrid Quantum-Classical Models
• DISCO: A Browser-Based Privacy-Preserving Framework for Distributed Collaborative Learning
• Privacy-Preserving Federated Learning from Partial Decryption Verifiable Threshold Multi-Client Functional Encryption
• The Capacity of Collusion-Resilient Decentralized Secure Aggregation with Groupwise Keys
• InfoDecom: Decomposing Information for Defending Against Privacy Leakage in Split Inference
• GraphToxin: Reconstructing Full Unlearned Graphs from Graph Unlearning
• Private Frequency Estimation Via Residue Number Systems
• On the Detectability of Active Gradient Inversion Attacks in Federated Learning
• Enhanced Privacy Leakage from Noise-Perturbed Gradients via Gradient-Guided Conditional Diffusion Models
• Experiences Building Enterprise-Level Privacy-Preserving Federated Learning to Power AI for Science
• LSHFed: Robust and Communication-Efficient Federated Learning with Locally-Sensitive Hashing Gradient Mapping
• Adversarial Node Placement in Decentralized Federated Learning: Maximum Spanning-Centrality Strategy and Performance Analysis
• SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning
• Differential Privacy: Gradient Leakage Attacks in Federated Learning Environments
• Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy
• GUIDE: Enhancing Gradient Inversion Attacks in Federated Learning with Denoising Models
• CLIP: Client-Side Invariant Pruning for Mitigating Stragglers in Secure Federated Learning
• SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition
• CoSIFL: Collaborative Secure and Incentivized Federated Learning with Differential Privacy
• Non-Linear Trajectory Modeling for Multi-Step Gradient Inversion Attacks in Federated Learning
• Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection
• No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks
• Advancing Practical Homomorphic Encryption for Federated Learning: Theoretical Guarantees and Efficiency Optimizations
• Rethinking Federated Learning Over the Air: The Blessing of Scaling Up
• Uncovering Privacy Vulnerabilities through Analytical Gradient Inversion Attacks
• ParaAegis: Parallel Protection for Flexible Privacy-preserved Federated Learning
• MAGIA: Sensing Per-Image Signals from Single-Round Averaged Gradients for Label-Inference-Free Gradient Inversion
• MAUI: Reconstructing Private Client Data in Federated Transfer Learning
• A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization
• Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
• Images in Motion?: A First Look into Video Leakage in Collaborative Deep Learning
• PLRV-O: Advancing Differentially Private Deep Learning via Privacy Loss Random Variable Optimization
• Benchmarking Robust Aggregation in Decentralized Gradient Marketplaces
• From Research to Reality: Feasibility of Gradient Inversion Attacks in Federated Learning
• FedUP: Efficient Pruning-based Federated Unlearning for Model Poisoning Attacks
• When Secure Aggregation Falls Short: Achieving Long-Term Privacy in Asynchronous Federated Learning for LEO Satellite Networks
• Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learning
• Adversarial Robustness in Distributed Quantum Machine Learning
• Deciphering the Interplay between Attack and Protection Complexity in Privacy-Preserving Federated Learning
• Multi-Hop Privacy Propagation for Differentially Private Federated Learning in Social Networks
• Per-element Secure Aggregation against Data Reconstruction Attacks in Federated Learning
• SelectiveShield: Lightweight Hybrid Defense Against Gradient Leakage in Federated Learning
• Evaluating Selective Encryption Against Gradient Inversion Attacks
• SenseCrypt: Sensitivity-guided Selective Homomorphic Encryption for Joint Federated Learning in Cross-Device Scenarios
• ASMR: Angular Support for Malfunctioning Client Resilience in Federated Learning
• IMU: Influence-guided Machine Unlearning
• Mitigating Persistent Client Dropout in Asynchronous Decentralized Federated Learning
• Information-Theoretic Decentralized Secure Aggregation with Passive Collusion Resilience
• Challenges of Trustworthy Federated Learning: What's Done, Current Trends and Remaining Work
• Evaluating the Dynamics of Membership Privacy in Deep Learning
• Efficient Machine Unlearning via Influence Approximation
• Proto-EVFL: Enhanced Vertical Federated Learning via Dual Prototype with Extremely Unaligned Data
• Uncovering Gradient Inversion Risks in Practical Language Model Training
• Scaling Decentralized Learning with FLock
• A Privacy-Centric Approach: Scalable and Secure Federated Learning Enabled by Hybrid Homomorphic Encryption
• FORTA: Byzantine-Resilient FL Aggregation via DFT-Guided Krum
• Hierarchical Secure Aggregation with Heterogeneous Security Constraints and Arbitrary User Collusion
• DRAGD: A Federated Unlearning Data Reconstruction Attack Based on Gradient Differences
• PROTEAN: Federated Intrusion Detection in Non-IID Environments through Prototype-Based Knowledge Sharing
• Efficient Unlearning with Privacy Guarantees
• Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages
• Privacy-Preserving Quantized Federated Learning with Diverse Precision
• Asymptotically Optimal Secure Aggregation for Wireless Federated Learning with Multiple Servers
• Privacy-Preserving Federated Learning Scheme with Mitigating Model Poisoning Attacks: Vulnerabilities and Countermeasures
• Embodied AI Agents: Modeling the World
• Equitable Federated Learning with NCA
• WallStreetFeds: Client-Specific Tokens as Investment Vehicles in Federated Learning
• Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning
• Hear No Evil: Detecting Gradient Leakage by Malicious Servers in Federated Learning
• SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation
• Topology-Aware Differential Privacy in Federated Learning
• Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning
• Federated Learning from Molecules to Processes: A Perspective
• Trustworthy Efficient Communication for Distributed Learning using LQ-SGD Algorithm
• PASS: Private Attributes Protection with Stochastic Data Substitution
• Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learning
• Byzantine Outside, Curious Inside: Reconstructing Data Through Malicious Updates
• A Quantitative Metric for Privacy Leakage in Federated Learning
• FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model
• Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models
• Similarity Weighted Aggregation with Global Differential Privacy for Federated Brain Lesion Segmentation
• Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation
• GCFL: A Gradient Correction-based Federated Learning Framework for Privacy-preserving CPSS
• Gradient Inversion Attacks on Parameter-Efficient Fine-Tuning
• PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
• DRAUN: An Algorithm-Agnostic Data Reconstruction Attack on Federated Unlearning Systems
• CSVAR: Enhancing Visual Privacy in Federated Learning via Adaptive Shuffling Against Overfitting
• Federated learning framework for collaborative remaining useful life prognostics: an aircraft engine case study
• Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models
• Label Leakage in Federated Inertial-based Human Activity Recognition
• LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environments
• Gradient Inversion Transcript: Leveraging Robust Generative Priors to Reconstruct Training Data from Gradient Leakage
• Cellular Traffic Prediction via Byzantine-robust Asynchronous Federated Learning
• EC-LDA : Label Distribution Inference Attack against Federated Graph Learning with Embedding Compression
• Efficient Privacy-Preserving Cross-Silo Federated Learning with Multi-Key Homomorphic Encryption
• MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning
• Cutting Through Privacy: A Hyperplane-Based Data Reconstruction Attack in Federated Learning
• LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures
• Potentials and Pitfalls of Applying Federated Learning in Hardware Assurance
• TrojanDam: Detection-Free Backdoor Defense in Federated Learning through Proactive Model Robustification utilizing OOD Data
• Differentially Private 2D Human Pose Estimation
• Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis
• Accelerating Differentially Private Federated Learning via Adaptive Extrapolation
• Personalizing Federated Learning for Hierarchical Edge Networks with Non-IID Data

Related

• Gradient Obfuscation Gives a False Sense of Security in Federated Learning
• Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learning
• When the Curious Abandon Honesty: Federated Learning Is Not Private
• Federated Learning Nodes Can Reconstruct Peers' Image Data
• Enhancing Privacy against Inversion Attacks in Federated Learning by using Mixing Gradients Strategies