2019/11/19 by Zhijie Lin, Zhou Zhao, Lin, Zhijie +7 · 4 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimedia (cs.MM) #Multimodal Machine Learning Applications #Video Analysis and Summarization #cs.CV #cs.LG #cs.MM
paper · pdf · doi:10.48550/arxiv.1911.08199
Accepted by AAAI 2020 as a full paper
openalex publication_date 2019/11/19 · arxiv created 2020/01/15 · arxiv updated 2020/01/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Video moment retrieval is to search the moment that is most relevant to the given natural language query. Existing methods are mostly trained in a fully-supervised setting, which requires the full annotations of temporal boundary for each query. However, manually labeling the annotations is actually time-consuming and expensive. In this paper, we propose a novel weakly-supervised moment retrieval framework requiring only coarse video-level annotations for training. Specifically, we devise a proposal generation module that aggregates the context information to generate and score all candidate proposals in one single pass. We then devise an algorithm that considers both exploitation and exploration to select top-K proposals. Next, we build a semantic completion module to measure the semantic similarity between the selected proposals and query, compute reward and provide feedbacks to the proposal generation module for scoring refinement. Experiments on the ActivityCaptions and Charades-STA demonstrate the effectiveness of our proposed method.