Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation

2019/02/26 by Heming Zhang, Shalini Ghosh, Zhang, Heming +11
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.1902.09818

IJCAI 2019

arxiv created 2019/08/14 · arxiv updated 2019/08/15

Abstract

The key challenge of generative Visual Dialogue (VD) systems is to respond to human queries with informative answers in natural and contiguous conversation flow. Traditional Maximum Likelihood Estimation (MLE)-based methods only learn from positive responses but ignore the negative responses, and consequently tend to yield safe or generic responses. To address this issue, we propose a novel training scheme in conjunction with weighted likelihood estimation (WLE) method. Furthermore, an adaptive multi-modal reasoning module is designed, to accommodate various dialogue scenarios automatically and select relevant information accordingly. The experimental results on the VisDial benchmark demonstrate the superiority of our proposed algorithm over other state-of-the-art approaches, with an improvement of 5.81% on recall@10.

Related

• Probabilistic framework for solving Visual Dialog
• DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Improving Generative Visual Dialog by Answering Diverse Questions