Average-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities

2012/02/18 by Eugene A. Feinberg, Pavlo O. Kasyanov, Feinberg, Eugene A. +3 · 7 citations
Computer Science · Decision Sciences · Economics, Econometrics and Finance · Mathematics · #90C40 #Decision-Making and Behavioral Economics #Economic theories and models #FOS: Mathematics #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #Stochastic processes and financial applications #math.OC #msc:90C40

paper · pdf · doi:10.48550/arxiv.1202.4122

26 pages

openalex publication_date 2012/02/18 · arxiv created 2012/02/19 · arxiv updated 2012/02/21 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

This paper presents sufficient conditions for the existence of stationary optimal policies for average-cost Markov Decision Processes with Borel state and action sets and with weakly continuous transition probabilities. The one-step cost functions may be unbounded, and action sets may be noncompact. The main contributions of this paper are: (i) general sufficient conditions for the existence of stationary discount-optimal and average-cost optimal policies and descriptions of properties of value functions and sets of optimal actions, (ii) a sufficient condition for the average-cost optimality of a stationary policy in the form of optimality inequalities, and (iii) approximations of average-cost optimal actions by discount-optimal actions.

Cited by

• Threshold-indexability of restless bandits with real interval state spaces: a performance-metric verification framework and long-run average analysis
• Long-Run Average Reward Maximization of A Regulated Regime-Switching Diffusion Model
• Berge's Theorem for Noncompact Image Sets
• Computing optimal policies for managing inventories with noisy observations
• On Pioneering Works of Albert Shiryaev on Markov Decision Processes and Some Later Developments
• Timely CPU Scheduling for Computation-intensive Status Updates
• Learning POMDPs with Linear Function Approximation and Finite Memory
• Rate-distortion Theory with Lower Semi-continuous Distortion on Noncompact Alphabets

Related

• Average optimality for continuous-time Markov decision processes under weak continuity conditions
• Average-Cost MDPs with Infinite State and Action Sets: New Sufficient Conditions for Optimality Inequalities and Equations
• On Markov Decision Processes with Borel Spaces and an Average Cost Criterion
• Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies
• Continuous-time Markov decision processes under the risk-sensitive average cost criterion