2020/11/02 by Majed El Helou, Helou, Majed El, Sabine Süsstrunk +1 · 1 citation
Computer Science · Engineering · #Advanced Image Processing Techniques #Image and Signal Denoising Methods #Advanced Image Fusion Techniques
paper · pdf · doi:10.48550/arxiv.2011.01406
Classic image-restoration algorithms use a variety of priors, either\nimplicitly or explicitly. Their priors are hand-designed and their\ncorresponding weights are heuristically assigned. Hence, deep learning methods\noften produce superior image restoration quality. Deep networks are, however,\ncapable of inducing strong and hardly predictable hallucinations. Networks\nimplicitly learn to be jointly faithful to the observed data while learning an\nimage prior; and the separation of original data and hallucinated data\ndownstream is then not possible. This limits their wide-spread adoption in\nimage restoration. Furthermore, it is often the hallucinated part that is\nvictim to degradation-model overfitting.\n We present an approach with decoupled network-prior based hallucination and\ndata fidelity terms. We refer to our framework as the Bayesian Integration of a\nGenerative Prior (BIGPrior). Our method is rooted in a Bayesian framework and\ntightly connected to classic restoration methods. In fact, it can be viewed as\na generalization of a large family of classic restoration algorithms. We use\nnetwork inversion to extract image prior information from a generative network.\nWe show that, on image colorization, inpainting and denoising, our framework\nconsistently improves the inversion results. Our method, though partly reliant\non the quality of the generative network inversion, is competitive with\nstate-of-the-art supervised and task-specific restoration methods. It also\nprovides an additional metric that sets forth the degree of prior reliance per\npixel relative to data fidelity.\n