Good Enough: Is it Worth Improving your Label Quality?

2025/05/27 by Jaus, Alexander, Marinov, Zdravko, Seibold, Constantin +3
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2505.20928

Abstract

Improving label quality in medical image segmentation is costly, but its benefits remain unclear. We systematically evaluate its impact using multiple pseudo-labeled versions of CT datasets, generated by models like nnU-Net, TotalSegmentator, and MedSAM. Our results show that while higher-quality labels improve in-domain performance, gains remain unclear if below a small threshold. For pre-training, label quality has minimal impact, suggesting that models rather transfer general concepts than detailed annotations. These findings provide guidance on when improving label quality is worth the effort.

Citations

• How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?
• AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking
• Impact of imperfect annotations on CNN training and performance for instance segmentation and classification in digital pathology
• Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
• Robust T-Loss for Medical Image Segmentation
• AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks
• Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
• STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training
• TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images
• AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
• Label noise in segmentation networks : mitigation must deal with bias
• Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotations
• nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Robustness study of noisy annotation in deep learning based medical image segmentation
• Quantitative Impact of Label Noise on the Quality of Segmentation of Brain Tumors on MRI scans
• Let's agree to disagree: learning highly debatable multirater labelling
• Imperfect Segmentation Labels: How Much Do They Matter?
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Related

• Quality Sentinel: Estimating Label Quality and Errors in Medical Segmentation Datasets
• Optimizing the Procedure of CT Segmentation Labeling
• Label Filling via Mixed Supervision for Medical Image Segmentation from Noisy Annotations
• Label noise in segmentation networks : mitigation must deal with bias
• Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directions