When Waiting is not an Option : Learning Options with a Deliberation\n Cost

2017/09/13 by Jean Harb, Harb, Jean, Pierre‐Luc Bacon +5 · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1709.04571

openalex publication_date 2017/09/13 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28

Abstract

Recent work has shown that temporally extended actions (options) can be\nlearned fully end-to-end as opposed to being specified in advance. While the\nproblem of "how" to learn options is increasingly well understood, the question\nof "what" good options should be has remained elusive. We formulate our answer\nto what "good" options should be in the bounded rationality framework (Simon,\n1957) through the notion of deliberation cost. We then derive practical\ngradient-based learning algorithms to implement this objective. Our results in\nthe Arcade Learning Environment (ALE) show increased performance and\ninterpretability.\n

Cited by

• Enhancing Hierarchical Reinforcement Learning through Change Point Detection in Time Series
• Toward Agents That Reason About Their Computation
• Towards General Computer Control with Hierarchical Agents and Multi-Level Action Spaces
• Scalable Option Learning in High-Throughput Environments

Related

• Learnings Options End-to-End for Continuous Action Tasks
• Learning with Options that Terminate Off-Policy
• The Option-Critic Architecture
• Attention Option-Critic
• Learning Abstract Options