Differentially-Private Decentralized Learning in Heterogeneous Multicast Networks

2025/09/25 by Ziaeddini, Amir, Yakimenka, Yauhen, Kliewer, Jörg
#FOS: Computer and information sciences #Information Theory (cs.IT)

paper · doi:10.48550/arxiv.2509.21688

Abstract

We propose a power-controlled differentially private decentralized learning algorithm designed for a set of clients aiming to collaboratively train a common learning model. The network is characterized by a row-stochastic adjacency matrix, which reflects different channel gains between the clients. In our privacy-preserving approach, both the transmit power for model updates and the level of injected Gaussian noise are jointly controlled to satisfy a given privacy and energy budget. We show that our proposed algorithm achieves a convergence rate of O(log T), where T is the horizon bound in the regret function. Furthermore, our numerical results confirm that our proposed algorithm outperforms existing works.

Citations

• Privacy Preserving Semi-Decentralized Mean Estimation over Intermittently-Connected Networks
• Decentralized Federated Learning: A Survey and Perspective
• Non-Coherent Over-the-Air Decentralized Gradient Descent
• Decentralized Federated Learning via Non-Coherent Over-the-Air Consensus
• Walk for Learning: A Random Walk Approach for Federated Learning from Heterogeneous Data
• Over-the-Air Decentralized Federated Learning
• Decentralized SGD with Over-the-Air Computation
• Federated Learning with Differential Privacy: Algorithms and Performance Analysis
• Federated Learning With Differential Privacy: Algorithms and Performance Analysis
• Push-Pull Gradient Methods for Distributed Optimization in Networks
• Distributed Optimization over Directed Graphs with Row Stochasticity and Constraint Regularity
• Distributed Convex Optimization with Inequality Constraints over Time-varying Unbalanced Digraphs
• Linear convergence in optimization over directed graphs with row-stochastic matrices
• Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying\n Directed Graphs
• Distributed optimization over time-varying directed graphs

Related

• LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learning
• Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints
• Privacy-preserving Decentralized Federated Learning over Time-varying Communication Graph
• PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution
• Dyn-D²P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee