Policy Gradient Methods for Reinforcement Learning with Function Approximation and Action-Dependent Baselines

2017/06/20 by Philip S. Thomas, Emma Brunskill, Thomas, Philip S. +1 · 4 citations
Computer Science · Engineering · #Adaptive Dynamic Programming Control #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Traffic control and management #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1706.06643

arxiv created 2017/06/20 · openalex publication_date 2017/06/20 · arxiv updated 2017/06/22 · openalex created_date 2017/06/30 · openalex updated_date 2026/07/28

Abstract

We show how an action-dependent baseline can be used by the policy gradient theorem using function approximation, originally presented with action-independent baselines by (Sutton et al. 2000).

Citations

• A Notation for Markov Decision Processes

Cited by

• Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMs
• SEM: Reinforcement Learning for Search-Efficient Large Language Models
• OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL
• KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation

Related

• The Mirage of Action-Dependent Baselines in Reinforcement Learning
• The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations
• Natural Policy Gradients In Reinforcement Learning Explained
• Policy Gradient using Weak Derivatives for Reinforcement Learning
• Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines