2019/06/04 by Janosch Haber, Haber, Janosch, Tim Baumgärtner +9 · 3 citations
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Language, Metaphor, and Cognition #Multimodal Machine Learning Applications #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.1906.01530
openalex publication_date 2019/06/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper introduces the PhotoBook dataset, a large-scale collection of\nvisually-grounded, task-oriented dialogues in English designed to investigate\nshared dialogue history accumulating during conversation. Taking inspiration\nfrom seminal work on dialogue analysis, we propose a data-collection task\nformulated as a collaborative game prompting two online participants to refer\nto images utilising both their visual context as well as previously established\nreferring expressions. We provide a detailed description of the task setup and\na thorough analysis of the 2,500 dialogues collected. To further illustrate the\nnovel features of the dataset, we propose a baseline model for reference\nresolution which uses a simple method to take into account shared information\naccumulated in a reference chain. Our results show that this information is\nparticularly important to resolve later descriptions and underline the need to\ndevelop more sophisticated models of common ground in dialogue interaction.\n