2020/05/27 by Sangjin Lee, Hyeongmin Lee, Lee, Sangjin +5
Computer Science · Engineering · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Processing Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2005.13194
openalex publication_date 2020/05/27 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
Video frame extrapolation is a task to predict future frames when the past\nframes are given. Unlike previous studies that usually have been focused on the\ndesign of modules or construction of networks, we propose a novel\nExtrapolative-Interpolative Cycle (EIC) loss using pre-trained frame\ninterpolation module to improve extrapolation performance. Cycle-consistency\nloss has been used for stable prediction between two function spaces in many\nvisual tasks. We formulate this cycle-consistency using two mapping functions;\nframe extrapolation and interpolation. Since it is easier to predict\nintermediate frames than to predict future frames in terms of the object\nocclusion and motion uncertainty, interpolation module can give guidance signal\neffectively for training the extrapolation function. EIC loss can be applied to\nany existing extrapolation algorithms and guarantee consistent prediction in\nthe short future as well as long future frames. Experimental results show that\nsimply adding EIC loss to the existing baseline increases extrapolation\nperformance on both UCF101 and KITTI datasets.\n