Reversible Markov decision processes and the Gaussian free field

2022/07/11 by Venkat Anantharam, Anantharam, Venkat · 1 citation
Computer Science · Decision Sciences · #FOS: Electrical engineering #FOS: Mathematics #Game Theory and Applications #Optimization and Control (math.OC) #Probability (math.PR) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2207.05217

openalex publication_date 2022/07/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings type dynamics. We characterize all discrete time reversible Markov decision processes with finite state and actions spaces. We show that policy iteration algorithm for finding an optimal policy can be significantly simplified Markov decision problems of this type. We also highlight the relation between the finite time evolution of the accrual of reward and the Gaussian free field associated to the controlled Markov chain.

Cited by

• Optimizing Predictive Maintenance in Intelligent Manufacturing: An Integrated FNO-DAE-GNN-PPO MDP Framework

Related

• Reductive MDPs: A Perspective Beyond Temporal Horizons
• Mean Field Markov Decision Processes
• An exact solution in Markov decision process with multiplicative rewards as a general framework
• Exponential Lower Bounds For Policy Iteration
• A Sliding-Window Algorithm for Markov Decision Processes with\n Arbitrarily Changing Rewards and Transitions