2023/08/19 by Yinda Chen, Wei Huang, Chen, Yinda +8
Biochemistry, Genetics and Molecular Biology · Materials Science · #Advanced Electron Microscopy Techniques and Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Electron and X-Ray Spectroscopy Techniques #FOS: Computer and information sciences #Machine Learning in Materials Science
paper · pdf · doi:10.48550/arxiv.2308.09917
openalex publication_date 2023/08/19 · openalex created_date 2023/08/23 · openalex updated_date 2026/07/28
Instance segmentation in electron microscopy (EM) volumes is tough due to complex shapes and sparse annotations. Self-supervised learning helps but still struggles with intricate visual patterns in EM. To address this, we propose a pretraining framework that enhances multiscale consistency in EM volumes. Our approach leverages a Siamese network architecture, integrating both strong and weak data augmentations to effectively extract multiscale features. We uphold voxel-level coherence by reconstructing the original input data from these augmented instances. Furthermore, we incorporate cross-attention mechanisms to facilitate fine-grained feature alignment between these augmentations. Finally, we apply contrastive learning techniques across a feature pyramid, allowing us to distill distinctive representations spanning various scales. After pretraining on four large-scale EM datasets, our framework significantly improves downstream tasks like neuron and mitochondria segmentation, especially with limited finetuning data. It effectively captures voxel and feature consistency, showing promise for learning transferable representations for EM analysis.