2020/12/02 by Jalal Arabneydi, Amir G. Aghdam, Arabneydi, Jalal +1
Business, Management and Accounting · Decision Sciences · Mathematics · #Advanced Queuing Theory Analysis #FOS: Mathematics #Game Theory and Applications #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2012.01020
openalex publication_date 2020/12/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper studies a large number of homogeneous Markov decision processes\nwhere the transition probabilities and costs are coupled in the empirical\ndistribution of states (also called mean-field). The state of each process is\nnot known to others, which means that the information structure is fully\ndecentralized. The objective is to minimize the average cost, defined as the\nempirical mean of individual costs, for which a sub-optimal solution is\nproposed. This solution does not depend on the number of processes, yet it\nconverges to the optimal solution of the so-called mean-field sharing as the\nnumber of processes tends to infinity. Under some mild conditions, it is shown\nthat the convergence rate of the proposed decentralized solution is\nproportional to the square root of the inverse of the number of processes.\nFinding this sub-optimal solution involves a non-smooth non-convex optimization\nproblem over an uncountable set, in general. To overcome this drawback, a\ncombinatorial optimization problem is introduced that achieves the same rate of\nconvergence.\n