Markov Decision Processes with Risk-Sensitive Criteria: An Overview

2023/11/12 by Nicole Bäuerle, Bäuerle, Nicole, Anna Jaśkiewicz +1 · 3 citations
Decision Sciences · #Decision-Making and Behavioral Economics #FOS: Economics and business #Risk Management (q-fin.RM) #Risk and Portfolio Optimization

paper · pdf · doi:10.48550/arxiv.2311.06896

openalex publication_date 2023/11/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk. This comprises the well-known entropic risk measure and Conditional Value-at-Risk. We restrict our considerations to stationary problems with an infinite time horizon. Conditions are given under which optimal policies exist and solution procedures are explained. We present both the theory when the Optimized Certainty Equivalent is applied recursively as well as the case where it is applied to the cumulated reward. Discounted as well as non-discounted models are reviewed

Cited by

• On the Bellman equation in recursive stochastic dynamic programming with the CES aggregator
• Policy Newton methods for Distortion Riskmetrics
• Markov Decision Processes with Value-at-Risk Criterion

Related

• Verification of Markov Decision Processes with Risk-Sensitive Measures
• Markov Decision Processes with Recursive Risk Measures
• Approximate Solutions To Constrained Risk-Sensitive Markov Decision Processes
• Efficient Risk-sensitive Planning via Entropic Risk Measures
• Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems with Markov Risk Measures