Perception Test 2023: A Summary of the First Challenge And Outcome

2023/12/20 by Heyward, Joseph, Carreira, João, Damen, Dima +2
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2312.13090

Abstract

The First Perception Test challenge was held as a half-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2023, with the goal of benchmarking state-of-the-art video models on the recently proposed Perception Test benchmark. The challenge had six tracks covering low-level and high-level tasks, with both a language and non-language interface, across video, audio, and text modalities, and covering: object tracking, point tracking, temporal action localisation, temporal sound localisation, multiple-choice video question-answering, and grounded video question-answering. We summarise in this report the task descriptions, metrics, baselines, and results.

Related

• Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark
• Perception Test: A Diagnostic Benchmark for Multimodal Video Models
• First Place Solution to the Multiple-choice Video QA Track of The Second Perception Test Challenge
• PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
• Pinpointing Trigger Moment for Grounded Video QA: Enhancing Spatio-temporal Grounding in Multimodal Large Language Models