Ghost-free High Dynamic Range Imaging with Context-aware Transformer

2022/08/10 by Zhen Liu, Liu, Zhen, Yinglong Wang +5 · 11 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Dynamic range #Engineering #FOS: Computer and information sciences #Ghosting #High dynamic range #High-dynamic-range imaging #Image Enhancement Techniques #Image and Signal Denoising Methods #Transformer #cs.CV

paper · pdf · doi:10.48550/arxiv.2208.05114

published in arXiv (Cornell University) (Cornell University) · ECCV 2022

arxiv created 2022/08/10 · openalex publication_date 2022/08/10 · arxiv updated 2022/08/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

High dynamic range (HDR) deghosting algorithms aim to generate ghost-free HDR images with realistic details. Restricted by the locality of the receptive field, existing CNN-based methods are typically prone to producing ghosting artifacts and intensity distortions in the presence of large motion and severe saturation. In this paper, we propose a novel Context-Aware Vision Transformer (CA-ViT) for ghost-free high dynamic range imaging. The CA-ViT is designed as a dual-branch architecture, which can jointly capture both global and local dependencies. Specifically, the global branch employs a window-based Transformer encoder to model long-range object movements and intensity variations to solve ghosting. For the local branch, we design a local context extractor (LCE) to capture short-range image features and use the channel attention mechanism to select informative local details across the extracted features to complement the global branch. By incorporating the CA-ViT as basic components, we further build the HDR-Transformer, a hierarchical network to reconstruct high-quality ghost-free HDR images. Extensive experiments on three benchmark datasets show that our approach outperforms state-of-the-art methods qualitatively and quantitatively with considerably reduced computational budgets. Codes are available at https://github.com/megvii-research/HDR-Transformer

Cited by

• GMODiff: One-Step Gain Map Refinement with Diffusion Priors for HDR Reconstruction
• Enabling High-Quality In-the-Wild Imaging from Severely Aberrated Metalens Bursts
• BurstDeflicker: A Benchmark Dataset for Flicker Removal in Dynamic Scenes
• Learning Efficient Meshflow and Optical Flow from Event Cameras
• EfficienT-HDR: An Efficient Transformer-Based Framework via Multi-Exposure Fusion for HDR Reconstruction
• PhysHDR: When Lighting Meets Materials and Scene Geometry in HDR Reconstruction
• AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes
• Capturing Stable HDR Videos Using a Dual-Camera System
• Enhancing Multi-Exposure High Dynamic Range Imaging with Overlapped Codebook for Improved Representation Learning
• iHDR: Iterative HDR Imaging with Arbitrary Number of Exposures
• S2R-HDR: A Large-Scale Rendered Dataset for HDR Fusion

Related

• Ghost-free High Dynamic Range Imaging via Hybrid CNN-Transformer and Structure Tensor
• High Dynamic Range Imaging with Context-aware Transformer
• Alignment-free HDR Deghosting with Semantics Consistent Transformer
• IFT: Image Fusion Transformer for Ghost-free High Dynamic Range Imaging
• HDRTransDC: High Dynamic Range Image Reconstruction with Transformer Deformation Convolution