Learning Dynamic Belief Graphs to Generalize on Text-Based Games

2020/02/21 by Ashutosh Adhikari, Adhikari, Ashutosh, Xingdi Yuan +17 · 4 citations
Computer Science · #Artificial Intelligence in Games #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2002.09127

openalex publication_date 2020/02/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challenge, and prior research has largely relied on hand-crafted structured representations and heuristics. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text. We propose a novel graph-aided transformer agent (GATA) that infers and updates latent belief graphs during planning to enable effective action selection by capturing the underlying game dynamics. GATA is trained using a combination of reinforcement and self-supervised learning. Our work demonstrates that the learned graph-based representations help agents converge to better policies than their text-only counterparts and facilitate effective generalization across game configurations. Experiments on 500+ unique games from the TextWorld suite show that our best agent outperforms text-based baselines by an average of 24.2%.

Citations

• Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
• Bidirectional Attention Flow for Machine Comprehension
• Neural Relational Inference for Interacting Systems
• GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
• Learning deep representations by mutual information estimation and maximization
• Go-Explore: a New Approach for Hard-Exploration Problems
• Language Understanding for Text-based Games Using Deep Reinforcement\n Learning
• TextWorld: A Learning Environment for Text-based Games
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Building Dynamic Knowledge Graphs from Text-based Games
• Layer Normalization
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Cited by

• Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
• Efficient Graph Understanding with LLMs via Structured Context Injection
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning
• CoEx -- Co-evolving World-model and Exploration

Related

• Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning
• Generalization in Text-based Games via Hierarchical Reinforcement Learning
• Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agents
• Graph Constrained Reinforcement Learning for Natural Language Action Spaces
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games