2016/05/26 by Alexander G. Anderson, Cory P. Berg, Anderson, Alexander G. +5 · 1 voice
Computer Science · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Neural and Evolutionary Computing (cs.NE) #cs.CV #cs.NE
paper · pdf · doi:10.48550/arxiv.1605.08153
11 pages, 5 figures
arxiv created 2016/05/26 · openalex publication_date 2016/05/26 · arxiv published 2016/05/26 · arxiv updated 2016/05/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A recent paper by Gatys et al. describes a method for rendering an image in the style of another image. First, they use convolutional neural network features to build a statistical model for the style of an image. Then they create a new image with the content of one image but the style statistics of another image. Here, we extend this method to render a movie in a given artistic style. The naive solution that independently renders each frame produces poor results because the features of the style move substantially from one frame to the next. The other naive method that initializes the optimization for the next frame using the rendered version of the previous frame also produces poor results because the features of the texture stay fixed relative to the frame of the movie instead of moving with objects in the scene. The main contribution of this paper is to use optical flow to initialize the style transfer optimization so that the texture features move with the objects in the video. Finally, we suggest a method to incorporate optical flow explicitly into the cost function.