2023/12/11 by Qiuhai Yan, Aiwen Jiang, Yan, Qiuhai +9 · 9 citations
Computer Science · #Advanced Image Processing Techniques #Artificial intelligence #Artificial neural network #Bridge (graph theory) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Convolution (computer science) #Enhanced Data Rates for GSM Evolution #FOS: Computer and information sciences #Focus (optics) #Image (mathematics) #Image Enhancement Techniques #Image and Signal Denoising Methods #Image editing #Image processing #Image restoration #Task (project management)
paper · pdf · doi:10.48550/arxiv.2312.06162
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/12/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Image restoration has always been a cutting-edge topic in the academic and industrial fields of computer vision. Since degradation signals are often random and diverse, "all-in-one" models that can do blind image restoration have been concerned in recent years. Early works require training specialized headers and tails to handle each degradation of concern, which are manually cumbersome. Recent works focus on learning visual prompts from data distribution to identify degradation type. However, the prompts employed in most of models are non-text, lacking sufficient emphasis on the importance of human-in-the-loop. In this paper, an effective textual prompt guided image restoration model has been proposed. In this model, task-specific BERT is fine-tuned to accurately understand user's instructions and generating textual prompt guidance. Depth-wise multi-head transposed attentions and gated convolution modules are designed to bridge the gap between textual prompts and visual features. The proposed model has innovatively introduced semantic prompts into low-level visual domain. It highlights the potential to provide a natural, precise, and controllable way to perform image restoration tasks. Extensive experiments have been done on public denoising, dehazing and deraining datasets. The experiment results demonstrate that, compared with popular state-of-the-art methods, the proposed model can obtain much more superior performance, achieving accurate recognition and removal of degradation without increasing model's complexity. Related source codes and data will be publicly available on github site https://github.com/MoTong-AI-studio/TextPromptIR.