2022/08/08 by Muammer Y. Yaman, Sergei V. Kalinin, Yaman, Muammer Y. +7 · 1 citation
Biochemistry, Genetics and Molecular Biology · Engineering · #Advanced Fluorescence Microscopy Techniques #Cell Image Analysis Techniques #Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Physical sciences #Optics (physics.optics) #Photoacoustic and Ultrasonic Imaging
paper · pdf · doi:10.48550/arxiv.2208.03861
openalex publication_date 2022/08/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We demonstrate the application of machine learning for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. We show that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that we can use hyperspectral darkfield microscopy to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.