2018/09/09 by Sohom Mukherjee, Arif Ahmed, Sk. Arif Ahmed +3 · 98 citations
Computer Science · Mathematics · #Artificial intelligence #Character (mathematics) #Computer science #Computer vision #Gesture #Hand Gesture Recognition Systems #Human Pose and Action Recognition #Initialization #Mathematics #Pattern recognition (psychology) #Segmentation #Speech recognition #Task (project management) #Tracking (education) #Video Surveillance and Tracking Methods #cs.CV
paper · pdf · doi:10.1016/j.eswa.2019.06.034
published in Expert Systems with Applications 136, 217-229 (Elsevier BV) · 32 pages, 10 figures, 2 tables. Submitted to Journal of Expert Systems with Applications
arxiv created 2018/09/09 · openalex publication_date 2019/06/17 · arxiv updated 2019/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 % while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 % using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems.