2021/11/20 by Ziyang Liu, Jingmeng Liu, Liu, Ziyang +7 · 2 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #Advanced Vision and Imaging #Artificial intelligence #Artificial neural network #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #FOS: Computer and information sciences #Feature (linguistics) #Flow network #Frame (networking) #Image (mathematics) #Mathematics #Object (grammar) #Optical flow #Pattern recognition (psychology) #Segmentation #Supervised learning #Video Surveillance and Tracking Methods #cs.CV
paper · pdf · doi:10.48550/arxiv.2111.10531
published in arXiv (Cornell University) (Cornell University) · Accepted by TIM (IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT)
arxiv created 2021/11/20 · openalex publication_date 2021/11/20 · arxiv updated 2021/11/23 · openalex created_date 2022/11/27 · openalex updated_date 2026/07/28
Semi-supervised video object segmentation (VOS) aims to segment a few moving objects in a video sequence, where these objects are specified by annotation of first frame. The optical flow has been considered in many existing semi-supervised VOS methods to improve the segmentation accuracy. However, the optical flow-based semi-supervised VOS methods cannot run in real time due to high complexity of optical flow estimation. A FAMINet, which consists of a feature extraction network (F), an appearance network (A), a motion network (M), and an integration network (I), is proposed in this study to address the abovementioned problem. The appearance network outputs an initial segmentation result based on static appearances of objects. The motion network estimates the optical flow via very few parameters, which are optimized rapidly by an online memorizing algorithm named relaxed steepest descent. The integration network refines the initial segmentation result using the optical flow. Extensive experiments demonstrate that the FAMINet outperforms other state-of-the-art semi-supervised VOS methods on the DAVIS and YouTube-VOS benchmarks, and it achieves a good trade-off between accuracy and efficiency. Our code is available at https://github.com/liuziyang123/FAMINet.