Stochastic dominance-constrained Markov decision processes

2012/06/20 by William B. Haskell, Haskell, William B., Rahul Jain +1 · 1 citation
Computer Science · Decision Sciences · Economics, Econometrics and Finance · #Economic theories and models #FOS: Mathematics #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #Risk and Portfolio Optimization

paper · pdf · doi:10.48550/arxiv.1206.4568

openalex publication_date 2012/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We are interested in risk constraints for infinite horizon discrete time Markov decision processes (MDPs). Starting with average reward MDPs, we show that increasing concave stochastic dominance constraints on the empirical distribution of reward lead to linear constraints on occupation measures. The optimal policy for the resulting class of dominance-constrained MDPs is obtained by solving a linear program. We compute the dual of this linear program to obtain average dynamic programming optimality equations that reflect the dominance constraint. In particular, a new pricing term appears in the optimality equations corresponding to the dominance constraint. We show that many types of stochastic orders can be used in place of the increasing concave stochastic order. We also carry out a parallel development for discounted reward MDPs with stochastic dominance constraints. The paper concludes with a portfolio optimization example.

Citations

• Constrained Markov Decision Processes

Cited by

• Relaxation of stochastic dominance constraints via optimal mass transport

Related

• Interval Dominance based Structural Results for Markov Decision Process
• Approximate Constrained Discounted Dynamic Programming with Uniform Feasibility and Optimality
• Higher-Order Stochastic Dominance Constraints in Optimization
• A convex programming approach for discrete-time Markov decision processes under the expected total reward criterion
• Mean Field Markov Decision Processes