2023/01/26 by Yi‐Fan Lin, Enlu Zhou, Lin, Yifan +1 · 1 citation
Decision Sciences · Energy · #Energy, Environment, and Transportation Policies #FOS: Electrical engineering #Risk and Portfolio Optimization #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2301.11415
openalex publication_date 2023/01/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider infinite-horizon Markov Decision Processes where parameters, such as transition probabilities, are unknown and estimated from data. The popular distributionally robust approach to addressing the parameter uncertainty can sometimes be overly conservative. In this paper, we utilize the recently proposed formulation, Bayesian risk Markov Decision Process (BR-MDP), to address parameter (or epistemic) uncertainty in MDPs. To solve the infinite-horizon BR-MDP with a class of convex risk measures, we propose a computationally efficient approach called approximate bilevel difference convex programming (ABDCP). The optimization is performed offline and produces the optimal policy that is represented as a finite state controller with desirable performance guarantees. We also demonstrate the empirical performance of the BR-MDP formulation and the proposed algorithm.