2020/12/07 by Joakim Johnander, Johnander, Joakim, Emil Brissman +5 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Visual Attention and Saliency Detection #Advanced Neural Network Applications
paper · pdf · doi:10.48550/arxiv.2012.03911
Most existing approaches to video instance segmentation comprise multiple modules that are heuristically combined to produce the final output. Formulating a purely learning-based method instead, which models both the temporal aspect as well as a generic track management required to solve the video instance segmentation task, is a highly challenging problem. In this work, we propose a novel learning formulation, where the entire video instance segmentation problem is modelled jointly. We fit a flexible model to our formulation that, with the help of a graph neural network, processes all available new information in each frame. Past information is considered and processed via a recurrent connection. We demonstrate the effectiveness of the proposed approach in comprehensive experiments. Our approach, operating at over 25 FPS, outperforms previous video real-time methods. We further conduct detailed ablative experiments that validate the different aspects of our approach.