Generalization to New Actions in Reinforcement Learning

2020/11/03 by Ayush Jain, Jain, Ayush, Andrew Szot +3 · 6 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2011.01928

openalex publication_date 2020/11/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices. However, standard reinforcement learning assumes a fixed set of actions and requires expensive retraining when given a new action set. To make learning agents more adaptable, we introduce the problem of zero-shot generalization to new actions. We propose a two-stage framework where the agent first infers action representations from action information acquired separately from the task. A policy flexible to varying action sets is then trained with generalization objectives. We benchmark generalization on sequential tasks, such as selecting from an unseen tool-set to solve physical reasoning puzzles and stacking towers with novel 3D shapes. Videos and code are available at https://sites.google.com/view/action-generalization

Cited by

• Training Language Models to Cooperate with Inference-Time Controllers
• Actor-Free Continuous Control via Structurally Maximizable Q-Functions
• Expanding the Action Space of LLMs to Reason Beyond Language
• Structure in Deep Reinforcement Learning: A Survey and Open Problems
• Learning to Synthesize Programs as Interpretable and Generalizable\n Policies
• Attacks and Defenses Against LLM Fingerprinting

Related

• Zero-Shot Action Generalization with Limited Observations
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• Explore to Generalize in Zero-Shot RL
• Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning
• Good Actions Succeed, Bad Actions Generalize: A Case Study on Why RL Generalizes Better