2022/03/29 by Xi Ye, Ye, Xi, Guillaume-Alexandre Bilodeau +1 · 2 citations
Computer Science · #Advanced Data Compression Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image and Video Quality Assessment #Video Coding and Compression Technologies
paper · pdf · doi:10.48550/arxiv.2203.15836
openalex publication_date 2022/03/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we propose a new Transformer block for video future frames prediction based on an efficient local spatial-temporal separation attention mechanism. Based on this new Transformer block, a fully autoregressive video future frames prediction Transformer is proposed. In addition, a non-autoregressive video prediction Transformer is also proposed to increase the inference speed and reduce the accumulated inference errors of its autoregressive counterpart. In order to avoid the prediction of very similar future frames, a contrastive feature loss is applied to maximize the mutual information between predicted and ground-truth future frame features. This work is the first that makes a formal comparison of the two types of attention-based video future frames prediction models over different scenarios. The proposed models reach a performance competitive with more complex state-of-the-art models. The source code is available at https://github.com/XiYe20/VPTR.