On the continuity of the projection mapping from strategic measures to occupation measures in absorbing Markov decision processes

2023/11/23 by Alexey Piunovskiy, Yi Zhang, Piunovskiy, Alexey +1 · 2 citations
Decision Sciences · Economics, Econometrics and Finance · #Economic theories and models #FOS: Mathematics #Game Theory and Applications #Game Theory and Voting Systems #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2311.14043

openalex publication_date 2023/11/23 · openalex created_date 2023/11/28 · openalex updated_date 2026/07/28

Abstract

In this paper, we prove the following assertion for an absorbing Markov decision process (MDP) with the given initial distribution, which is also assumed to be semi-continuous: the continuity of the projection mapping from the space of strategic measures to the space of occupation measures, both endowed with their weak topologies, is equivalent to the MDP model being uniformly absorbing. An example demonstrates, among other interesting scenarios, that for an absorbing (but not uniformly absorbing) semi-continuous MDP with the given initial distribution, the space of occupation measures can fail to be compact in the weak topology.

Cited by

• Absorbing Markov Decision Processes: Geometric Properties and Sufficiency of Finite Mixtures of Deterministic Policies
• On the Feinberg-Piunovskiy Theorem and its extension to chattering policies

Related

• Further remarks on absorbing Markov decision processes
• A note on weak compactness of occupation measures for an absorbing Markov decision process
• Absorbing Markov Decision Processes
• Absorbing Markov Decision Processes: Geometric Properties and Sufficiency of Finite Mixtures of Deterministic Policies
• Extreme occupation measures in Markov decision processes with a cemetery