A variational formula for risk-sensitive reward

2015/01/04 by Venkatachalam Anantharam, Anantharam, Venkatachalam, Vivek Shripad Borkar +1 · 2 citations
Computer Science · Mathematics · #93E20 #94A15 #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Optimization and Control (math.OC) #Probability (math.PR) #cs.IT #math.IT #math.OC #math.PR #msc:93E20 #msc:94A15

paper · pdf · doi:10.48550/arxiv.1501.00676

35 pages

arxiv created 2015/01/04 · arxiv updated 2015/01/06

Abstract

We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact metric state and action spaces, extending a formula of Donsker and Varadhan for the Perron-Frobenius eigenvalue of a positive operator. This leads to a concave maximization formulation of the problem of determining this optimal growth rate.

Cited by

• Policy Newton methods for Distortion Riskmetrics
• Markov control of continuous time Markov processes with long run functionals by time discretization

Related

• Linear and dynamic programs for risk-sensitive cost minimization
• Mean Field Markov Decision Processes
• Risk-sensitive control of continuous time Markov chains
• Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity
• On risk-sensitive piecewise deterministic Markov decision processes