Video Captioning via Hierarchical Reinforcement Learning

2017/11/29 by Xin Wang, Wenhu Chen, Wang, Xin +7 · 6 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications #Video Analysis and Summarization

paper · pdf · doi:10.48550/arxiv.1711.11135

openalex publication_date 2017/11/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Video captioning is the task of automatically generating a textual description of the actions in a video. Although previous work (e.g. sequence-to-sequence model) has shown promising results in abstracting a coarse description of a short video, it is still very challenging to caption a video containing multiple fine-grained actions with a detailed description. This paper aims to address the challenge by proposing a novel hierarchical reinforcement learning framework for video captioning, where a high-level Manager module learns to design sub-goals and a low-level Worker module recognizes the primitive actions to fulfill the sub-goal. With this compositional framework to reinforce video captioning at different levels, our approach significantly outperforms all the baseline methods on a newly introduced large-scale dataset for fine-grained video captioning. Furthermore, our non-ensemble model has already achieved the state-of-the-art results on the widely-used MSR-VTT dataset.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Long Short-Term Memory
• Bidirectional recurrent neural networks
• Continuous control with deep reinforcement learning
• Sequence Level Training with Recurrent Neural Networks
• Reinforcement Learning Neural Turing Machines - Revised
• Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• An Actor-Critic Algorithm for Sequence Prediction
• Deep Residual Learning for Image Recognition
• Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
• Sequence to Sequence -- Video to Text
• Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning
• Learning Spatiotemporal Features with 3D Convolutional Networks
• Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding

Cited by

• VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimization
• IntentVCNet: Bridging Spatio-Temporal Gaps for Intention-Oriented Controllable Video Captioning
• An Attempt towards Interpretable Audio-Visual Video Captioning
• Towards Diverse Paragraph Captioning for Untrimmed Videos
• Dense Video Captioning using Graph-based Sentence Summarization
• Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization

Related

• METEOR Guided Divergence for Video Captioning
• Hierarchical Modular Network for Video Captioning
• Pretrained Image-Text Models are Secretly Video Captioners
• VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking
• Streamlined Dense Video Captioning