Entropic Risk Constrained Soft-Robust Policy Optimization

2020/06/20 by Russel, Reazul Hasan, Behzadian, Bahram, Petrik, Marek
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2006.11679

Abstract

Having a perfect model to compute the optimal policy is often infeasible in reinforcement learning. It is important in high-stakes domains to quantify and manage risk induced by model uncertainties. Entropic risk measure is an exponential utility-based convex risk measure that satisfies many reasonable properties. In this paper, we propose an entropic risk constrained policy gradient and actor-critic algorithms that are risk-averse to the model uncertainty. We demonstrate the usefulness of our algorithms on several problem domains.

Related

• Soft-Robust Actor-Critic Policy-Gradient
• Tight Sample Complexity Bounds for Entropic Best Policy Identification
• Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts
• On the Global Convergence of Risk-Averse Natural Policy Gradient Methods with Expected Conditional Risk Measures
• Efficient Risk-sensitive Planning via Entropic Risk Measures