Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning

2018/12/04 by Prithviraj Ammanabrolu, Mark Riedl, Mark O. Riedl +2 · 1 voice · 8 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Digital Games and Media #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1812.01628

Proceedings of NAACL-HLT 2019

openalex publication_date 2018/12/04 · arxiv published 2018/12/04 · arxiv created 2019/03/25 · arxiv updated 2019/03/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Text-based adventure games provide a platform on which to explore reinforcement learning in the context of a combinatorial action space, such as natural language. We present a deep reinforcement learning architecture that represents the game state as a knowledge graph which is learned during exploration. This graph is used to prune the action space, enabling more efficient exploration. The question of which action to take can be reduced to a question-answering task, a form of transfer learning that pre-trains certain parts of our architecture. In experiments using the TextWorld framework, we show that our proposed technique can learn a control policy faster than baseline alternatives. We have also open-sourced our code at https://github.com/rajammanabrolu/KG-DQN.

Citations

• Sequence to Sequence Learning with Neural Networks
• The Natural Language Decathlon: Multitask Learning as Question Answering
• TextWorld: A Learning Environment for Text-based Games

Cited by

• Web World Models
• ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluation
• Vejde: A Framework for Inductive Deep Reinforcement Learning Based on Factor Graph Color Refinement
• AI Reasoning for Wireless Communications and Networking: A Survey and Perspectives
• Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games
• Learning Natural Language Generation from Scratch
• Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments
• Learning Natural Language Constraints for Safe Reinforcement Learning of Language Agents

Discussions

• Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning [hn, 3 points, 0 comments]

Related

• Transfer in Deep Reinforcement Learning using Knowledge Graphs
• Graph Constrained Reinforcement Learning for Natural Language Action Spaces
• Algorithmic Improvements for Deep Reinforcement Learning applied to Interactive Fiction
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games
• How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds