2022/06/27 by Zhan Chen, Sicheng Li, Chen, Zhan +7 · 31 citations
Computer Science · Engineering · #Action recognition #Algorithm #Anomaly Detection Techniques and Applications #Artificial intelligence #Artificial neural network #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Convolution (computer science) #Convolutional neural network #FOS: Computer and information sciences #Gait Recognition and Analysis #Graph #Human Pose and Action Recognition #Pattern recognition (psychology) #RGB color model #Theoretical computer science #cs.CV
paper · pdf · doi:10.48550/arxiv.2206.13028
published in arXiv (Cornell University) (Cornell University) · 10 pages, 4 figures, accepted by AAAI 2021
arxiv created 2022/06/27 · openalex publication_date 2022/06/27 · arxiv updated 2022/06/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Graph convolutional networks have been widely used for skeleton-based action recognition due to their excellent modeling ability of non-Euclidean data. As the graph convolution is a local operation, it can only utilize the short-range joint dependencies and short-term trajectory but fails to directly model the distant joints relations and long-range temporal information that are vital to distinguishing various actions. To solve this problem, we present a multi-scale spatial graph convolution (MS-GC) module and a multi-scale temporal graph convolution (MT-GC) module to enrich the receptive field of the model in spatial and temporal dimensions. Concretely, the MS-GC and MT-GC modules decompose the corresponding local graph convolution into a set of sub-graph convolution, forming a hierarchical residual architecture. Without introducing additional parameters, the features will be processed with a series of sub-graph convolutions, and each node could complete multiple spatial and temporal aggregations with its neighborhoods. The final equivalent receptive field is accordingly enlarged, which is capable of capturing both short- and long-range dependencies in spatial and temporal domains. By coupling these two modules as a basic block, we further propose a multi-scale spatial temporal graph convolutional network (MST-GCN), which stacks multiple blocks to learn effective motion representations for action recognition. The proposed MST-GCN achieves remarkable performance on three challenging benchmark datasets, NTU RGB+D, NTU-120 RGB+D and Kinetics-Skeleton, for skeleton-based action recognition.