2021/07/30 by Waleed Tahir, Hao Wang, Lei Tian
Computer Science · Engineering · Physics and Astronomy · #Adaptability #Artificial intelligence #Artificial neural network #Computer Graphics and Visualization Techniques #Computer science #Deep learning #Digital Holography and Microscopy #Machine learning #Network architecture #Optics #Physics #Random lasers and scattering media #Robustness (evolution) #Scattering #eess.IV #physics.optics
paper · pdf · doi:10.1038/s41377-022-00730-x
published in Light Science & Applications 11(1), 42 (Springer Nature)
openalex publication_date 2021/07/30 · arxiv created 2022/02/03 · arxiv updated 2022/02/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Deep learning has been broadly applied to imaging in scattering applications. A common framework is to train a descattering network for image recovery by removing scattering artifacts. To achieve the best results on a broad spectrum of scattering conditions, individual "expert" networks need to be trained for each condition. However, the expert's performance sharply degrades when the testing condition differs from the training. An alternative brute-force approach is to train a "generalist" network using data from diverse scattering conditions. It generally requires a larger network to encapsulate the diversity in the data and a sufficiently large training set to avoid overfitting. Here, we propose an adaptive learning framework, termed dynamic synthesis network (DSN), which dynamically adjusts the model weights and adapts to different scattering conditions. The adaptability is achieved by a novel "mixture of experts" architecture that enables dynamically synthesizing a network by blending multiple experts using a gating network. We demonstrate the DSN in holographic 3D particle imaging for a variety of scattering conditions. We show in simulation that our DSN provides generalization across a continuum of scattering conditions. In addition, we show that by training the DSN entirely on simulated data, the network can generalize to experiments and achieve robust 3D descattering. We expect the same concept can find many other applications, such as denoising and imaging in scattering media. Broadly, our dynamic synthesis framework opens up a new paradigm for designing highly adaptive deep learning and computational imaging techniques.