Q-Learning in Regularized Mean-field Games

2020/03/24 by Berkay Anahtarcı, Anahtarci, Berkay, Can Deha Karıksız +3 · 5 citations
Biochemistry, Genetics and Molecular Biology · #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Receptor Mechanisms and Signaling #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2003.12151

openalex publication_date 2020/03/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we introduce a regularized mean-field game and study learning of this game under an infinite-horizon discounted reward function. Regularization is introduced by adding a strongly concave regularization function to the one-stage reward function in the classical mean-field game model. We establish a value iteration based learning algorithm to this regularized mean-field game using fitted Q-learning. The regularization term in general makes reinforcement learning algorithm more robust to the system components. Moreover, it enables us to establish error analysis of the learning algorithm without imposing restrictive convexity assumptions on the system components, which are needed in the absence of a regularization term.

Cited by

• Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces
• Entropy Regularization in Mean-Field Games of Optimal Stopping
• Learning in Stackelberg Markov Games
• Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning
• Approximation of Discrete-Time Infinite-Horizon Mean-Field Equilibria via Finite-Horizon Mean-Field Equilibria

Related

• Unified Reinforcement Q-Learning for Mean Field Game and Control Problems
• Convergence of Multi-Scale Reinforcement Q-Learning Algorithms for Mean Field Game and Control Problems
• Reinforcement Learning for Mean Field Games, with Applications to Economics
• Learning in games via reinforcement and regularization
• Learning Mean-Field Games