FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative

2025/12/23 by Mohammadreza Rostami, Rostami, Mohammadreza, Solmaz S. Kia +1
Computer Science · #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #Cryptography and Data Security

paper · doi:10.48550/arxiv.2512.20814

Abstract

This paper introduces FedMPDD (Federated Learning via Multi-Projected Directional Derivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of FedMPDD is to encode each client's high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from O(d) to O(m), where m ≪ d. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that FedMPDD converges at a rate of O(1/√(K)), matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.

Citations

• FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
• GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Search
• SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning
• Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reduction
• Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
• pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
• First-Order Dynamic Optimization for Streaming Convex Costs
• Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approach
• Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agents
• Scaling Forward Gradient With Local Losses
• On Privacy and Personalization in Cross-Silo Federated Learning
• Gradients without Backpropagation
• Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?
• Communication Efficiency in Federated Learning: Achievements and Challenges
• LoRA: Low-Rank Adaptation of Large Language Models
• DP-SIGNSGD: When Efficiency Meets Privacy and Robustness
• See through Gradients: Image Batch Recovery via GradInversion
• FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• Personalized Federated Learning with Moreau Envelopes
• LDP-Fed: Federated Learning with Local Differential Privacy
• Local Differential Privacy based Federated Learning for Internet of Things
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Wireless Federated Learning with Local Differential Privacy
• iDLG: Improved Deep Leakage from Gradients
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Advances and Open Problems in Federated Learning
• Federated Learning with Differential Privacy: Algorithms and Performance Analysis
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Federated Learning for Wireless Communications: Motivation,\n Opportunities and Challenges
• Deep Learning With Edge Computing: A Review
• Deep Leakage from Gradients
• Natural Compression for Distributed Deep Learning
• Communication-efficient distributed SGD with Sketching
• Robust and Communication-Efficient Federated Learning from Non-IID Data
• Error Feedback Fixes SignSGD and other Gradient Compression Schemes
• The Convergence of Sparsified Gradient Methods
• cpSGD: Communication-efficient and differentially-private distributed SGD
• Local SGD Converges Fast and Communicates Little
• Exploiting Unintended Feature Leakage in Collaborative Learning
• signSGD: Compressed Optimisation for Non-Convex Problems
• Soft Weight-Sharing for Neural Network Compression
• Distributed Mean Estimation with Limited Communication
• A Secure Control Framework for Resource-Limited Adversaries

Related

• FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection
• sqSGD: Locally Private and Communication Efficient Federated Learning
• SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
• FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection
• Harnessing Sparsification in Federated Learning: A Secure, Efficient, and Differentially Private Realization