A note on weak compactness of occupation measures for an absorbing Markov decision process

2024/02/16 by François Dufour, Dufour, François, Tomás Prieto-Rumeau +1
Decision Sciences · #90C40 #FOS: Mathematics #Optimization and Control (math.OC) #Probability (math.PR) #Simulation Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2402.10672

openalex publication_date 2024/02/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

We consider an absorbing Markov decision process with Borel state and action spaces. We study conditions under which the MDP is uniformly absorbing and the set of occupation measures of the MDP is compact in the usual weak topology. These include suitable continuity requirements on the transition kernel and conditions on the dynamics of the system at the boundary of the absorbing set. We generalize previously known results and give an answer to some conjectures that have been mentioned in the related literature.

Related

• Absorbing Markov Decision Processes
• Further remarks on absorbing Markov decision processes
• On the continuity of the projection mapping from strategic measures to occupation measures in absorbing Markov decision processes
• Extreme occupation measures in Markov decision processes with a cemetery
• Absorbing Markov Decision Processes: Geometric Properties and Sufficiency of Finite Mixtures of Deterministic Policies