FedFMS: Exploring Federated Foundation Models for Medical Image Segmentation

2024/03/08 by Yu-xi Liu, Liu, Yuxi, Guibo Luo +3 · 5 citations
Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #Distributed #FOS: Computer and information sciences #FOS: Electrical engineering #I.2.11 #I.4.6 #Image and Video Processing (eess.IV) #Medical Imaging and Analysis #Parallel #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2403.05408

openalex publication_date 2024/03/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Medical image segmentation is crucial for clinical diagnosis. The Segmentation Anything Model (SAM) serves as a powerful foundation model for visual segmentation and can be adapted for medical image segmentation. However, medical imaging data typically contain privacy-sensitive information, making it challenging to train foundation models with centralized storage and sharing. To date, there are few foundation models tailored for medical image deployment within the federated learning framework, and the segmentation performance, as well as the efficiency of communication and training, remain unexplored. In response to these issues, we developed Federated Foundation models for Medical image Segmentation (FedFMS), which includes the Federated SAM (FedSAM) and a communication and training-efficient Federated SAM with Medical SAM Adapter (FedMSA). Comprehensive experiments on diverse datasets are conducted to investigate the performance disparities between centralized training and federated learning across various configurations of FedFMS. The experiments revealed that FedFMS could achieve performance comparable to models trained via centralized training methods while maintaining privacy. Furthermore, FedMSA demonstrated the potential to enhance communication and training efficiency. Our model implementation codes are available at https://github.com/LIU-YUXI/FedFMS.

Cited by

• FRIEREN: Federated Learning with Vision-Language Regularization for Segmentation
• pFedSAM: Personalized Federated Learning of Segment Anything Model for Medical Image Segmentation
• Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights
• FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models
• A New One-Shot Federated Learning Framework for Medical Imaging Classification with Feature-Guided Rectified Flow and Knowledge Distillation

Related

• pFedSAM: Personalized Federated Learning of Segment Anything Model for Medical Image Segmentation
• Where to Begin? From Random to Foundation Model Instructed Initialization in Federated Learning for Medical Image Segmentation
• FedSCA: Federated Tuning with Similarity-guided Collaborative Aggregation for Heterogeneous Medical Image Segmentation
• Adversarial Versus Federated: An Adversarial Learning based Multi-Modality Cross-Domain Federated Medical Segmentation
• Federated Alternate Training (FAT): Leveraging Unannotated Data Silos in Federated Segmentation for Medical Imaging