Hyperbolically-Discounted Reinforcement Learning on Reward-Punishment Framework

2021/06/03 by Taisuke Kobayashi, Kobayashi, Taisuke
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Auction Theory and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2106.01516

openalex publication_date 2021/06/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a new reinforcement learning with hyperbolic discounting. Combining a new temporal difference error with the hyperbolic discounting in recursive manner and reward-punishment framework, a new scheme to learn the optimal policy is derived. In simulations, it is found that the proposal outperforms the standard reinforcement learning, although the performance depends on the design of reward and punishment. In addition, the averages of discount factors w.r.t. reward and punishment are different from each other, like a sign effect in animal behaviors.

Related

• Hyperbolic Discounting and Learning over Multiple Horizons
• Reinforcement Learning with Non-Exponential Discounting
• Reinforcement Learning with Quasi-Hyperbolic Discounting
• Examining average and discounted reward optimality criteria in reinforcement learning
• Discounted Reinforcement Learning Is Not an Optimization Problem