Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

2016/08/22 by Pichao Wang, Wanqing Li, Wang, Pichao +9 · 3 citations
Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Gait Recognition and Analysis #Hand Gesture Recognition Systems #Human Pose and Action Recognition

paper · pdf · doi:10.48550/arxiv.1608.06338

openalex publication_date 2016/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

This paper addresses the problem of continuous gesture recognition from sequences of depth maps using convolutional neutral networks (ConvNets). The proposed method first segments individual gestures from a depth sequence based on quantity of movement (QOM). For each segmented gesture, an Improved Depth Motion Map (IDMM), which converts the depth sequence into one image, is constructed and fed to a ConvNet for recognition. The IDMM effectively encodes both spatial and temporal information and allows the fine-tuning with existing ConvNet models for classification without introducing millions of parameters to learn. The proposed method is evaluated on the Large-scale Continuous Gesture Recognition of the ChaLearn Looking at People (LAP) challenge 2016. It achieved the performance of 0.2655 (Mean Jaccard Index) and ranked 3^rd place in this challenge.

Citations

• Two-Stream Convolutional Networks for Action Recognition in Videos
• ImageNet classification with deep convolutional neural networks

Cited by

• RGB-D-based Human Motion Recognition with Deep Learning: A Survey
• Scene Flow to Action Map: A New Representation for RGB-D based Action Recognition with Convolutional Neural Networks
• Large-scale Isolated Gesture Recognition Using Convolutional Neural Networks

Related

• Large-scale Isolated Gesture Recognition Using Convolutional Neural Networks
• Depth Pooling Based Large-scale 3D Action Recognition with Convolutional Neural Networks
• A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture Recognition System
• HGR-Net: A Fusion Network for Hand Gesture Segmentation and Recognition
• Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition