Federated Deep Reinforcement Learning

2019/01/24 by Hankz Hankui Zhuo, Zhuo, Hankz Hankui, Wenfeng Feng +7 · 5 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #Transportation and Mobility Innovations

paper · pdf · doi:10.48550/arxiv.1901.08277

openalex publication_date 2019/01/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.

Citations

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning

Cited by

• Convergence Guarantees for Federated SARSA with Local Training and Heterogeneous Agents
• Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments
• On the Convergence of FedAvg on Non-IID Data
• Edge Intelligence: Architectures, Challenges, and Applications
• Communication-Efficient Local Decentralized SGD Methods

Related

• Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning
• FedHPD: Heterogeneous Federated Reinforcement Learning via Policy Distillation
• FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning
• A Privacy-preserving Distributed Training Framework for Cooperative Multi-agent Deep Reinforcement Learning
• Locally Private Distributed Reinforcement Learning