Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems with Markov Risk Measures

2022/09/26 by Shiping Shao, Abhishek Gupta, Shao, Shiping +3
Decision Sciences · #FOS: Electrical engineering #FOS: Mathematics #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2209.12937

openalex publication_date 2022/09/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. We further establish the robustness of the risk-sensitive optimal policies to modeling errors. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.

Related

• Markov Decision Processes with Risk-Sensitive Criteria: An Overview
• Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
• Robust Markov Decision Processes
• Robust Markov Decision Process: Beyond Rectangularity
• Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity