2020/04/14 by Dian Shao, Yue Zhao, Shao, Dian +5 · 44 citations
Computer Science · #Action (physics) #Artificial intelligence #Class (philosophy) #Computer science #Event (particle physics) #Granularity #Hierarchy #Human Pose and Action Recognition #Machine learning #Multimodal Machine Learning Applications #Natural language processing #Parsing #Quality (philosophy) #Set (abstract data type) #Video Analysis and Summarization #cs.CV
paper · pdf · doi:10.48550/arxiv.2004.06704
published in arXiv (Cornell University) (Cornell University) · CVPR 2020 Oral (3 strong accepts); Project page: https://sdolivia.github.io/FineGym/
arxiv created 2020/04/14 · openalex publication_date 2020/04/14 · arxiv updated 2020/04/15 · openalex created_date 2020/04/24 · openalex updated_date 2026/08/06
On public benchmarks, current action recognition techniques have achieved great success. However, when used in real-world applications, e.g. sport analysis, which requires the capability of parsing an activity into phases and differentiating between subtly different actions, their performances remain far from being satisfactory. To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnastic videos. Compared to existing action recognition datasets, FineGym is distinguished in richness, quality, and diversity. In particular, it provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy. For example, a "balance beam" event will be annotated as a sequence of elementary sub-actions derived from five sets: "leap-jump-hop", "beam-turns", "flight-salto", "flight-handspring", and "dismount", where the sub-action in each set will be further annotated with finely defined class labels. This new level of granularity presents significant challenges for action recognition, e.g. how to parse the temporal structures from a coherent action, and how to distinguish between subtly different action classes. We systematically investigate representative methods on this dataset and obtain a number of interesting findings. We hope this dataset could advance research towards action understanding.