2021/01/10 by Peter Washington, Aaron Kline, Washington, Peter +15
Health Professions · Neuroscience · Social Sciences · #Assistive Technology in Communication and Mobility #Autism Spectrum Disorder Research #Child Development and Digital Technology #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)
paper · pdf · doi:10.48550/arxiv.2101.03478
openalex publication_date 2021/01/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Activity recognition computer vision algorithms can be used to detect the\npresence of autism-related behaviors, including what are termed "restricted and\nrepetitive behaviors", or stimming, by diagnostic instruments. The limited data\nthat exist in this domain are usually recorded with a handheld camera which can\nbe shaky or even moving, posing a challenge for traditional feature\nrepresentation approaches for activity detection which mistakenly capture the\ncamera's motion as a feature. To address these issues, we first document the\nadvantages and limitations of current feature representation techniques for\nactivity recognition when applied to head banging detection. We then propose a\nfeature representation consisting exclusively of head pose keypoints. We create\na computer vision classifier for detecting head banging in home videos using a\ntime-distributed convolutional neural network (CNN) in which a single CNN\nextracts features from each frame in the input sequence, and these extracted\nfeatures are fed as input to a long short-term memory (LSTM) network. On the\nbinary task of predicting head banging and no head banging within videos from\nthe Self Stimulatory Behaviour Dataset (SSBD), we reach a mean F1-score of\n90.77% using 3-fold cross validation (with individual fold F1-scores of 83.3%,\n89.0%, and 100.0%) when ensuring that no child who appeared in the train set\nwas in the test set for all folds. This work documents a successful technique\nfor training a computer vision classifier which can detect human motion with\nfew training examples and even when the camera recording the source clips is\nunstable. The general methods described here can be applied by designers and\ndevelopers of interactive systems towards other human motion and pose\nclassification problems used in mobile and ubiquitous interactive systems.\n