Guiding Reinforcement Learning Exploration Using Natural Language

2017/07/26 by Harrison, Brent, Ehsan, Upol, Riedl, Mark O.
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1707.08616

Abstract

In this work we present a technique to use natural language to help reinforcement learning generalize to unseen environments. This technique uses neural machine translation, specifically the use of encoder-decoder networks, to learn associations between natural language behavior descriptions and state-action information. We then use this learned model to guide agent exploration using a modified version of policy shaping to make it more effective at learning in unseen environments. We evaluate this technique using the popular arcade game, Frogger, under ideal and non-ideal conditions. This evaluation shows that our modified policy shaping algorithm improves over a Q-learning agent as well as a baseline version of policy shaping.

Related

• Influencing Reinforcement Learning through Natural Language Guidance
• Language is Power: Representing States Using Natural Language in Reinforcement Learning
• PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards
• Grounding Language for Transfer in Deep Reinforcement Learning
• A Survey of Reinforcement Learning Informed by Natural Language