2021/12/11 by Bing Zhou, Zhou, Bing, Matias Aiskovich +3
Computer Science · #FOS: Computer and information sciences #Hand Gesture Recognition Systems #Human-Computer Interaction (cs.HC) #Music and Audio Processing #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2112.05986
openalex publication_date 2021/12/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Hand gesture recognition attracts great attention for interaction since it is intuitive and natural to perform. In this paper, we explore a novel method for interaction by using bone-conducted sound generated by finger movements while performing gestures. We design a set of gestures that generate unique sound features, and capture the resulting sound from the wrist using a commodity microphone. Next, we design a sound event detector and a recognition model to classify the gestures. Our system achieves an overall accuracy of 90.13% in quiet environments and 85.79% under noisy conditions. This promising technology can be deployed on existing smartwatches as a low power service at no additional cost, and can be used for interaction in augmented and virtual reality applications.