Imperfect Segmentation Labels: How Much Do They Matter?

2018/06/12 by Nicholas Heller, Joshua Dean, Heller, Nicholas +3 · 4 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Medical Image Segmentation Techniques #cs.CV

paper · pdf · doi:10.48550/arxiv.1806.04618

9 pages, 3 figures, Accepted at MICCAI LABELS 2018

openalex publication_date 2018/06/12 · arxiv created 2018/09/24 · arxiv updated 2018/09/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Labeled datasets for semantic segmentation are imperfect, especially in medical imaging where borders are often subtle or ill-defined. Little work has been done to analyze the effect that label errors have on the performance of segmentation methodologies. Here we present a large-scale study of model performance in the presence of varying types and degrees of error in training data. We trained U-Net, SegNet, and FCN32 several times for liver segmentation with 10 different modes of ground-truth perturbation. Our results show that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures. We also found that each architecture was very robust to non-boundary-localized errors, suggesting that boundary-localized errors are fundamentally different and more challenging problem than random label errors in a classification setting.

Citations

• Adam: A Method for Stochastic Optimization
• SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation

Cited by

• Learning to Detect Label Errors by Making Them: A Method for Segmentation and Object Detection Datasets
• Good Enough: Is it Worth Improving your Label Quality?
• Improving Training on Noisy Stuctured Labels
• Adversarial Robustness of Deep Learning Models for Inland Water Body Segmentation from SAR Images

Related

• Label noise in segmentation networks : mitigation must deal with bias
• Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotations
• Estimating label quality and errors in semantic segmentation data via any model
• Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification
• Disentangling Human Error from the Ground Truth in Segmentation of Medical Images