2024/09/26 by Xin Cai, Cai, Xin, Zhiyuan You +9 · 4 citations
Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image Processing Techniques and Applications #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Optical Coherence Tomography Applications #Photoacoustic and Ultrasonic Imaging #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2409.17996
openalex publication_date 2024/09/26 · openalex created_date 2024/11/01 · openalex updated_date 2026/07/28
Lensless cameras offer significant advantages in size, weight, and cost compared to traditional lens-based systems. Without a focusing lens, lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, current algorithms struggle with inaccurate forward imaging models and insufficient priors to reconstruct high-quality images. To overcome these limitations, we introduce a novel two-stage approach for consistent and photorealistic lensless image reconstruction. The first stage of our approach ensures data consistency by focusing on accurately reconstructing the low-frequency content with a spatially varying deconvolution method that adjusts to changes in the Point Spread Function (PSF) across the camera's field of view. The second stage enhances photorealism by incorporating a generative prior from pre-trained diffusion models. By conditioning on the low-frequency content retrieved in the first stage, the diffusion model effectively reconstructs the high-frequency details that are typically lost in the lensless imaging process, while also maintaining image fidelity. Our method achieves a superior balance between data fidelity and visual quality compared to existing methods, as demonstrated with two popular lensless systems, PhlatCam and DiffuserCam. Project website: https://phocolens.github.io/.