2019/05/10 by Nilay Shrivastava, Shrivastava, Nilay, Astitwa Saxena +10 · 2 citations
Computer Science · Engineering · #Artificial intelligence #Artificial neural network #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Convolution (computer science) #Convolutional neural network #Deep learning #Engineering #FOS: Computer and information sciences #Footprint #Hand Gesture Recognition Systems #Indoor and Outdoor Localization Technologies #Memory footprint #Mobile device #Quantization (signal processing) #Speech and Audio Processing #Speech recognition #Task (project management) #cs.CL #cs.CV
paper · pdf · doi:10.48550/arxiv.1905.03968
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2019/05/10 · arxiv created 2019/06/05 · arxiv updated 2019/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Visual speech recognition (VSR) is the task of recognizing spoken language from video input only, without any audio. VSR has many applications as an assistive technology, especially if it could be deployed in mobile devices and embedded systems. The need of intensive computational resources and large memory footprint are two of the major obstacles in developing neural network models for VSR in a resource constrained environment. We propose a novel end-to-end deep neural network architecture for word level VSR called MobiVSR with a design parameter that aids in balancing the model's accuracy and parameter count. We use depthwise-separable 3D convolution for the first time in the domain of VSR and show how it makes our model efficient. MobiVSR achieves an accuracy of 73% on a challenging Lip Reading in the Wild dataset with 6 times fewer parameters and 20 times lesser memory footprint than the current state of the art. MobiVSR can also be compressed to 6 MB by applying post training quantization.