Frequency Domain Transformer Networks for Video Prediction

2019/03/01 by Hafez Farazi, Farazi, Hafez, Sven Behnke +1 · 1 citation
Computer Science · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image and Signal Denoising Methods #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1903.00271

openalex publication_date 2019/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The task of video prediction is forecasting the next frames given some previous frames. Despite much recent progress, this task is still challenging mainly due to high nonlinearity in the spatial domain. To address this issue, we propose a novel architecture, Frequency Domain Transformer Network (FDTN), which is an end-to-end learnable model that estimates and uses the transformations of the signal in the frequency domain. Experimental evaluations show that this approach can outperform some widely used video prediction methods like Video Ladder Network (VLN) and Predictive Gated Pyramids (PGP).

Citations

• Transformation-Based Models of Video Sequences
• PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning

Cited by

• VideoPCDNet: Video Parsing and Prediction with Phase Correlation Networks

Related

• Local Frequency Domain Transformer Networks for Video Prediction
• Motion Segmentation using Frequency Domain Transformer Networks
• Video Prediction Transformers without Recurrence or Convolution
• TKN: Transformer-based Keypoint Prediction Network For Real-time Video Prediction
• Fourier-based Video Prediction through Relational Object Motion