MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolution

2022/05/10 by Chong Mou, Yanze Wu, Mou, Chong +9 · 4 citations
Computer Science · Engineering · Physics and Astronomy · #Advanced Image Processing Techniques #Advanced Optical Sensing Technologies #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Optical Systems and Laser Technology #cs.CV #eess.IV #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2205.05065

Accepted by ECCV 2022. Code is available at: https://github.com/TencentARC/MM-RealSR

openalex publication_date 2022/05/10 · arxiv created 2022/07/27 · arxiv updated 2022/07/28 · openalex created_date 2022/07/30 · openalex updated_date 2026/07/28

Abstract

Interactive image restoration aims to restore images by adjusting several controlling coefficients, which determine the restoration strength. Existing methods are restricted in learning the controllable functions under the supervision of known degradation types and levels. They usually suffer from a severe performance drop when the real degradation is different from their assumptions. Such a limitation is due to the complexity of real-world degradations, which can not provide explicit supervision to the interactive modulation during training. However, how to realize the interactive modulation in real-world super-resolution has not yet been studied. In this work, we present a Metric Learning based Interactive Modulation for Real-World Super-Resolution (MM-RealSR). Specifically, we propose an unsupervised degradation estimation strategy to estimate the degradation level in real-world scenarios. Instead of using known degradation levels as explicit supervision to the interactive mechanism, we propose a metric learning strategy to map the unquantifiable degradation levels in real-world scenarios to a metric space, which is trained in an unsupervised manner. Moreover, we introduce an anchor point strategy in the metric learning process to normalize the distribution of metric space. Extensive experiments demonstrate that the proposed MM-RealSR achieves excellent modulation and restoration performance in real-world super-resolution. Codes are available at https://github.com/TencentARC/MM-RealSR.

Cited by

• Zero-shot Adaptation of Stable Diffusion via Plug-in Hierarchical Degradation Representation for Real-World Super-Resolution
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution
• A Systematic Investigation on Deep Learning-Based Omnidirectional Image and Video Super-Resolution
• Controlled Data Rebalancing in Multi-Task Learning for Real-World Image Super-Resolution

Related

• Interactive Multi-Dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration
• Learning Multiple Probabilistic Degradation Generators for Unsupervised Real World Image Super Resolution
• Towards Realistic Data Generation for Real-World Super-Resolution
• Toward Real-world Image Super-resolution via Hardware-based Adaptive Degradation Models
• RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought