2023/03/23 by Chenyang Lyu, Lyu, Chenyang, Manh-Duy Nguyen +9 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Multimodal Machine Learning Applications #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2303.16761
openalex publication_date 2023/03/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent years have witnessed an increasing amount of dialogue/conversation on the web especially on social media. That inspires the development of dialogue-based retrieval, in which retrieving videos based on dialogue is of increasing interest for recommendation systems. Different from other video retrieval tasks, dialogue-to-video retrieval uses structured queries in the form of user-generated dialogue as the search descriptor. We present a novel dialogue-to-video retrieval system, incorporating structured conversational information. Experiments conducted on the AVSD dataset show that our proposed approach using plain-text queries improves over the previous counterpart model by 15.8% on R@1. Furthermore, our approach using dialogue as a query, improves retrieval performance by 4.2%, 6.2%, 8.6% on R@1, R@5 and R@10 and outperforms the state-of-the-art model by 0.7%, 3.6% and 6.0% on R@1, R@5 and R@10 respectively.