Average-Cost Markov Decision Processes with Weakly Continuous Transition\n Probabilities

2012/02/18 by Eugene A. Feinberg, Pavlo O. Kasyanov, Feinberg, Eugene A. +3 · 4 citations
Computer Science · Decision Sciences · Economics, Econometrics and Finance · #90C40 #Decision-Making and Behavioral Economics #Economic theories and models #FOS: Mathematics #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #Stochastic processes and financial applications

paper · pdf · doi:10.48550/arxiv.1202.4122

openalex publication_date 2012/02/18 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

This paper presents sufficient conditions for the existence of stationary\noptimal policies for average-cost Markov Decision Processes with Borel state\nand action sets and with weakly continuous transition probabilities. The\none-step cost functions may be unbounded, and action sets may be noncompact.\nThe main contributions of this paper are: (i) general sufficient conditions for\nthe existence of stationary discount-optimal and average-cost optimal policies\nand descriptions of properties of value functions and sets of optimal actions,\n(ii) a sufficient condition for the average-cost optimality of a stationary\npolicy in the form of optimality inequalities, and (iii) approximations of\naverage-cost optimal actions by discount-optimal actions.\n

Cited by

• Threshold-indexability of restless bandits with real interval state spaces: a performance-metric verification framework and long-run average analysis
• Long-Run Average Reward Maximization of A Regulated Regime-Switching Diffusion Model
• Berge's Theorem for Noncompact Image Sets
• Computing optimal policies for managing inventories with noisy observations

Related

• Average optimality for continuous-time Markov decision processes under weak continuity conditions
• Average-Cost MDPs with Infinite State and Action Sets: New Sufficient Conditions for Optimality Inequalities and Equations
• On Markov Decision Processes with Borel Spaces and an Average Cost Criterion
• Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies
• Continuous-time Markov decision processes under the risk-sensitive average cost criterion