2019/07/28 by Pushkar Shukla, Shukla, Pushkar, Carlos Elmadjian +9
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Language, Metaphor, and Cognition #Multimedia (cs.MM) #Multimodal Machine Learning Applications #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1907.12021
openalex publication_date 2019/07/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The ability to engage in goal-oriented conversations has allowed humans to\ngain knowledge, reduce uncertainty, and perform tasks more efficiently.\nArtificial agents, however, are still far behind humans in having goal-driven\nconversations. In this work, we focus on the task of goal-oriented visual\ndialogue, aiming to automatically generate a series of questions about an image\nwith a single objective. This task is challenging since these questions must\nnot only be consistent with a strategy to achieve a goal, but also consider the\ncontextual information in the image. We propose an end-to-end goal-oriented\nvisual dialogue system, that combines reinforcement learning with regularized\ninformation gain. Unlike previous approaches that have been proposed for the\ntask, our work is motivated by the Rational Speech Act framework, which models\nthe process of human inquiry to reach a goal. We test the two versions of our\nmodel on the GuessWhat?! dataset, obtaining significant results that outperform\nthe current state-of-the-art models in the task of generating questions to find\nan undisclosed object in an image.\n