2021/11/19 by Ties van Rozendaal, Johann Brehmer, van Rozendaal, Ties +7 · 1 citation
Computer Science · #Advanced Data Compression Techniques #Advanced Image Processing Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Signal Denoising Methods #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Video Coding and Compression Technologies #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2111.10302
openalex publication_date 2021/11/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce a video compression algorithm based on instance-adaptive learning. On each video sequence to be transmitted, we finetune a pretrained compression model. The optimal parameters are transmitted to the receiver along with the latent code. By entropy-coding the parameter updates under a suitable mixture model prior, we ensure that the network parameters can be encoded efficiently. This instance-adaptive compression algorithm is agnostic about the choice of base model and has the potential to improve any neural video codec. On UVG, HEVC, and Xiph datasets, our codec improves the performance of a scale-space flow model by between 21% and 27% BD-rate savings, and that of a state-of-the-art B-frame model by 17 to 20% BD-rate savings. We also demonstrate that instance-adaptive finetuning improves the robustness to domain shift. Finally, our approach reduces the capacity requirements of compression models. We show that it enables a competitive performance even after reducing the network size by 70%.