CraftAssist: A Framework for Dialogue-enabled Interactive Agents

2019/07/19 by Jonathan Gray, Kavya Srinet, Gray, Jonathan +15 · 4 citations
Computer Science · #AI in Service Interactions #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1907.08584

openalex publication_date 2019/07/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper describes an implementation of a bot assistant in Minecraft, and the tools and platform allowing players to interact with the bot and to record those interactions. The purpose of building such an assistant is to facilitate the study of agents that can complete tasks specified by dialogue, and eventually, to learn from dialogue interactions.

Citations

• Playing Atari with Deep Reinforcement Learning
• Language to Logical Form with Neural Attention
• Control of Memory, Active Perception, and Action in Minecraft
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Data Recombination for Neural Semantic Parsing
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• AI2-THOR: An Interactive 3D Environment for Visual AI
• CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant
• VQA: Visual Question Answering

Cited by

• Can Current Agents Close the Discovery-to-Application Gap? A Case Study in Minecraft
• Talking-to-Build: How LLM-Assisted Interface Shapes Player Performance and Experience in Minecraft
• CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended World
• Learning Adaptive Language Interfaces through Decomposition

Related

• Why Build an Assistant in Minecraft?
• Improving Agent Interactions in Virtual Environments with Language Models
• BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
• Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback
• Converse: A Tree-Based Modular Task-Oriented Dialogue System