2024/03/12 by Yiran Wang, Xiao Li, Wang, Yiran +1 · 2 citations
Biochemistry, Genetics and Molecular Biology · Engineering · Materials Science · #Advanced Electron Microscopy Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Electron and X-Ray Spectroscopy Techniques #FOS: Computer and information sciences #FOS: Electrical engineering #Image Processing Techniques and Applications #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2403.07951
openalex publication_date 2024/03/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
It has been shown that traditional deep learning methods for electronic microscopy segmentation usually suffer from low transferability when samples and annotations are limited, while large-scale vision foundation models are more robust when transferring between different domains but facing sub-optimal improvement under fine-tuning. In this work, we present a new few-shot domain adaptation framework SAMDA, which combines the Segment Anything Model(SAM) with nnUNet in the embedding space to achieve high transferability and accuracy. Specifically, we choose the Unet-based network as the "expert" component to learn segmentation features efficiently and design a SAM-based adaptation module as the "generic" component for domain transfer. By amalgamating the "generic" and "expert" components, we mitigate the modality imbalance in the complex pre-training knowledge inherent to large-scale Vision Foundation models and the challenge of transferability inherent to traditional neural networks. The effectiveness of our model is evaluated on two electron microscopic image datasets with different modalities for mitochondria segmentation, which improves the dice coefficient on the target domain by 6.7%. Also, the SAM-based adaptor performs significantly better with only a single annotated image than the 10-shot domain adaptation on nnUNet. We further verify our model on four MRI datasets from different sources to prove its generalization ability.