Quantitative propagation of chaos for mean field Markov decision process with common noise

2022/07/26 by Médéric Motte, Huyên Pham, Motte, Médéric +1 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Diffusion and Search Dynamics #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Probability (math.PR) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2207.12738

openalex publication_date 2022/07/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We investigate propagation of chaos for mean field Markov Decision Process with common noise (CMKV-MDP), and when the optimization is performed over randomized open-loop controls on infinite horizon. We first state a rate of convergence of order M_N^γ, where M_N is the mean rate of convergence in Wasserstein distance of the empirical measure, and γ∈ (0,1] is an explicit constant, in the limit of the value functions of N-agent control problem with asymmetric open-loop controls, towards the value function of CMKV-MDP. Furthermore, we show how to explicitly construct (ε+O(M_N^γ))-optimal policies for the N-agent model from ε-optimal policies for the CMKV-MDP. Our approach relies on sharp comparison between the Bellman operators in the N-agent problem and the CMKV-MDP, and fine coupling of empirical measures.

Citations

• On the rate of convergence in Wasserstein distance of the empirical\n measure

Cited by

• Robust mean-field control under common noise uncertainty

Related

• Mean-field Markov decision processes with common noise and open-loop controls
• Robust mean-field control under common noise uncertainty
• Extended mean field control problem: a propagation of chaos result
• Limit theory for mean-field control problems with common noise adapted controls
• Finite state N-agent and mean field control problems