2020/12/07 by Joakim Johnander, Johnander, Joakim, Emil Brissman +5 · 8 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial intelligence #Artificial neural network #Computer science #Frame (networking) #Graph #Machine learning #Pattern recognition (psychology) #Recurrent neural network #Segmentation #Task (project management) #Theoretical computer science #Visual Attention and Saliency Detection #cs.CV
paper · pdf · doi:10.48550/arxiv.2012.03911
published in arXiv (Cornell University) (Cornell University)
arxiv created 2020/12/07 · openalex publication_date 2020/12/07 · arxiv updated 2020/12/08 · openalex created_date 2020/12/21 · openalex updated_date 2026/07/28
Most existing approaches to video instance segmentation comprise multiple modules that are heuristically combined to produce the final output. Formulating a purely learning-based method instead, which models both the temporal aspect as well as a generic track management required to solve the video instance segmentation task, is a highly challenging problem. In this work, we propose a novel learning formulation, where the entire video instance segmentation problem is modelled jointly. We fit a flexible model to our formulation that, with the help of a graph neural network, processes all available new information in each frame. Past information is considered and processed via a recurrent connection. We demonstrate the effectiveness of the proposed approach in comprehensive experiments. Our approach, operating at over 25 FPS, outperforms previous video real-time methods. We further conduct detailed ablative experiments that validate the different aspects of our approach.