Mobile Video Object Detection with Temporally-Aware Feature Maps

2017/11/17 by Mason Liu, Liu, Mason, Menglong Zhu +1 · 3 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.1711.06368

openalex publication_date 2017/11/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces an online model for object detection in videos designed to run in real-time on low-powered mobile and embedded devices. Our approach combines fast single-image object detection with convolutional long short term memory (LSTM) layers to create an interweaved recurrent-convolutional architecture. Additionally, we propose an efficient Bottleneck-LSTM layer that significantly reduces computational cost compared to regular LSTMs. Our network achieves temporal awareness by using Bottleneck-LSTMs to refine and propagate feature maps across frames. This approach is substantially faster than existing detection methods in video, outperforming the fastest single-frame models in model size and computational cost while attaining accuracy comparable to much more expensive single-frame models on the Imagenet VID 2015 dataset. Our model reaches a real-time inference speed of up to 15 FPS on a mobile CPU.

Citations

• ImageNet Large Scale Visual Recognition Challenge
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Speeding-up Convolutional Neural Networks Using Fine-tuned\n CP-Decomposition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Deep Residual Learning for Image Recognition
• Quantized Convolutional Neural Networks for Mobile Devices
• Xception: Deep Learning with Depthwise Separable Convolutions
• Rich feature hierarchies for accurate object detection and semantic\n segmentation

Cited by

• Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks
• MVP: Motion Vector Propagation for Zero-Shot Video Object Detection
• FASTER Recurrent Networks for Efficient Video Classification

Related

• Looking Fast and Slow: Memory-Guided Mobile Video Object Detection
• Memory Warps for Learning Long-Term Online Video Representations
• Temporally Identity-Aware SSD with Attentional LSTM
• Towards High Performance Video Object Detection for Mobiles
• ApproxDet: Content and Contention-Aware Approximate Object Detection for Mobiles