Distributed Nonconvex Optimization with Double Privacy Protection and Exact Convergence

2025/11/04 by Zichong Ou, Ou, Zichong, Dandan Wang +5
Computer Science · #Distributed Control Multi-Agent Systems #FOS: Mathematics #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2511.02283

openalex publication_date 2025/11/04 · openalex created_date 2025/11/06 · openalex updated_date 2026/07/28

Abstract

Motivated by the pervasive lack of privacy protection in existing distributed nonconvex optimization methods, this paper proposes a decentralized proximal primal-dual algorithm enabling double protection of privacy (DPP²) for minimizing nonconvex sum-utility functions over multi-agent networks, which ensures zero leakage of critical local information during inter-agent communications. We develop a two-tier privacy protection mechanism that first merges the primal and dual variables by means of a variable transformation, followed by embedding an additional random perturbation to further obfuscate the transmitted information. We theoretically establish that DPP² ensures differential privacy for local objectives while achieving exact convergence under nonconvex settings. Specifically, DPP² converges sublinearly to a stationary point and attains a linear convergence rate under the additional Polyak-Łojasiewicz (P-Ł) condition. Finally, a numerical example demonstrates the superiority of DPP² over a number of state-of-the-art algorithms, showcasing the faster, exact convergence achieved by DPP² under the same level of differential privacy.

Citations

• PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Bounds
• Robust Constrained Consensus and Inequality-constrained Distributed Optimization with Guaranteed Differential Privacy and Accurate Convergence
• Differentially Private and Communication-Efficient Distributed Nonconvex Optimization Algorithms
• Compressed Differentially Private Distributed Optimization with Linear Convergence
• DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learning
• Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy
• Decentralized Stochastic Optimization with Inherent Privacy Protection
• A Unifying Approximate Method of Multipliers for Distributed Composite Optimization
• Linear Convergence of First- and Zeroth-Order Primal-Dual Algorithms for Distributed Nonconvex Optimization
• Deep Learning for Distributed Optimization: Applications to Wireless Resource Management
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Optimal algorithms for smooth and strongly convex distributed optimization in networks
• On Nonconvex Decentralized Gradient Descent
• Distributed Nonconvex Multiagent Optimization Over Time-Varying Networks
• NEXT: In-Network Nonconvex Optimization
• A Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization
• Privacy Preservation in Distributed Subgradient Optimization Algorithms
• Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems
• Differentially Private Distributed Optimization

Related

• Differentially Private and Communication-Efficient Distributed Nonconvex Optimization Algorithms
• Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy
• Guaranteed Privacy of Distributed Nonconvex Optimization via Mixed-Monotone Functional Perturbations
• Privacy-Preserving Distributed Online Mirror Descent for Nonconvex Optimization
• Privacy-Preserving Distributed Zeroth-Order Optimization