2013/12/30 by Sergio Valcárcel Macua, Macua, Sergio Valcarcel, Jianshu Chen +5 · 1 citation
Computer Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #Distributed #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Opinion Dynamics and Social Influence #Parallel #Reinforcement Learning in Robotics #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.1312.7606
openalex publication_date 2013/12/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are following. The proposed distributed strategy is efficient, with linear complexity in both computation time and memory footprint. We provide a mean-square-error performance analysis and establish convergence under constant step-size updates, which endow the network with continuous learning capabilities. The results show a clear gain from cooperation: when the individual agents can estimate the solution, cooperation increases stability and reduces bias and variance of the prediction error; but, more importantly, the network is able to approach the optimal solution even when none of the individual agents can (e.g., when the individual behavior policies restrict each agent to sample a small portion of the state space).