2019/04/29 by Dong Liu, Yue Li, Jianping Lin +2 · 150 citations
Computer Science · Engineering · #Advanced Image Processing Techniques #Artificial intelligence #Codec #Coding (social sciences) #Computer science #Convolutional neural network #Deep learning #Encoder #Image and Signal Denoising Methods #Machine learning #Source code #Telecommunications #Video Coding and Compression Technologies #cs.MM #eess.IV
paper · pdf · doi:10.1145/3368405
published in ACM Computing Surveys 53(1), 1-35 (Association for Computing Machinery)
arxiv created 2019/04/29 · openalex created_date 2019/05/03 · openalex publication_date 2020/02/06 · arxiv updated 2020/03/13 · openalex updated_date 2026/08/05
The past decade has witnessed the great success of deep learning in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. We review the representative works about using deep learning for image/video coding, an actively developing research area since 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks, and deep network-based coding tools that shall be used within traditional coding schemes. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding and transform coding, respectively. For deep tools, there have been several techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter and CNN-based block adaptive resolution coding. The source code of DLVC has been released for future research.