2023/08/19 by Canzhe Zhao, Zhao, Canzhe, Yanjie Ze +6
Computer Science · Engineering · Social Sciences · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy, Security, and Data Protection #Privacy-Preserving Technologies in Data #Wireless Communication Security Techniques
paper · pdf · doi:10.48550/arxiv.2308.09902
openalex publication_date 2023/08/19 · openalex created_date 2023/08/23 · openalex updated_date 2026/07/28
Communication lays the foundation for cooperation in human society and in multi-agent reinforcement learning (MARL). Humans also desire to maintain their privacy when communicating with others, yet such privacy concern has not been considered in existing works in MARL. To this end, we propose the differentially private multi-agent communication (DPMAC) algorithm, which protects the sensitive information of individual agents by equipping each agent with a local message sender with rigorous (ε, δ)-differential privacy (DP) guarantee. In contrast to directly perturbing the messages with predefined DP noise as commonly done in privacy-preserving scenarios, we adopt a stochastic message sender for each agent respectively and incorporate the DP requirement into the sender, which automatically adjusts the learned message distribution to alleviate the instability caused by DP noise. Further, we prove the existence of a Nash equilibrium in cooperative MARL with privacy-preserving communication, which suggests that this problem is game-theoretically learnable. Extensive experiments demonstrate a clear advantage of DPMAC over baseline methods in privacy-preserving scenarios.