2021/09/08 by Ziyi Chen, Yi Zhou, Chen, Ziyi +5 · 5 citations
Computer Science · Engineering · #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Smart Grid Security and Resilience
paper · pdf · doi:10.48550/arxiv.2109.03699
openalex publication_date 2021/09/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy. However, existing decentralized AC algorithms either do not preserve the privacy of agents or are not sample and communication-efficient. In this work, we develop two decentralized AC and natural AC (NAC) algorithms that are private, and sample and communication-efficient. In both algorithms, agents share noisy information to preserve privacy and adopt mini-batch updates to improve sample and communication efficiency. Particularly for decentralized NAC, we develop a decentralized Markovian SGD algorithm with an adaptive mini-batch size to efficiently compute the natural policy gradient. Under Markovian sampling and linear function approximation, we prove the proposed decentralized AC and NAC algorithms achieve the state-of-the-art sample complexities O(ε-2ln(ε-1)) and O(ε-3ln(ε-1)), respectively, and the same small communication complexity O(ε-1ln(ε-1)). Numerical experiments demonstrate that the proposed algorithms achieve lower sample and communication complexities than the existing decentralized AC algorithm.