Variance Adjusted Actor Critic Algorithms

2013/10/14 by Tamar, Aviv, Mannor, Shie
#FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.1310.3697

Abstract

We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost surely to a locally optimal point of the objective function.

Related

• Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPs
• Variance Penalized On-Policy and Off-Policy Actor-Critic
• Variance Reduction in Actor Critic Methods (ACM)
• Variational Actor-Critic Algorithms
• A Finite-Sample Analysis of an Actor-Critic Algorithm for Mean-Variance Optimization in a Discounted MDP