2025/05/23 by Shenghui Chen, Po-han Li, Chen, Shenghui +5
Computer Science · Social Sciences · #Computer Vision and Pattern Recognition (cs.CV) #Digital Rights Management and Security #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Information Theory (cs.IT) #Multimedia Communication and Technology #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2505.17423
openalex publication_date 2025/05/23 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28
Many decision-making tasks, where both accuracy and efficiency matter, still require human supervision. For example, tasks like traffic officers reviewing hour-long dashcam footage or researchers screening conference videos can benefit from concise summaries that reduce cognitive load and save time. Yet current vision-language models (VLMs) often produce verbose, redundant outputs that hinder task performance. Existing video caption evaluation depends on costly human annotations and overlooks the summaries' utility in downstream tasks. We address these gaps with Video-to-text Information Bottleneck Evaluation (VIBE), an annotation-free method that scores VLM outputs using two metrics: grounding (how well the summary aligns with visual content) and utility (how informative it is for the task). VIBE selects from randomly sampled VLM outputs by ranking them according to the two scores to support effective human decision-making. Human studies on LearningPaper24, SUTD-TrafficQA, and LongVideoBench show that summaries selected by VIBE consistently improve performance-boosting task accuracy by up to 61.23% and reducing response time by 75.77% compared to naive VLM summaries or raw video.