Robust Semantic Segmentation with Superpixel-Mix

2021/08/02 by Gianni Franchi, Nacim Belkhir, Franchi, Gianni +11 · 6 citations
Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Consistency (knowledge bases) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Image segmentation #Machine Learning (stat.ML) #Machine learning #Pattern recognition (psychology) #Reliability (semiconductor) #Robustness (evolution) #Segmentation #cs.AI #cs.CV #stat.ML

paper · pdf · doi:10.48550/arxiv.2108.00968

published in arXiv (Cornell University) (Cornell University) · Accepted to BMVC2021

openalex publication_date 2021/08/02 · arxiv created 2021/10/21 · arxiv updated 2021/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Along with predictive performance and runtime speed, reliability is a key requirement for real-world semantic segmentation. Reliability encompasses robustness, predictive uncertainty and reduced bias. To improve reliability, we introduce Superpixel-mix, a new superpixel-based data augmentation method with teacher-student consistency training. Unlike other mixing-based augmentation techniques, mixing superpixels between images is aware of object boundaries, while yielding consistent gains in segmentation accuracy. Our proposed technique achieves state-of-the-art results in semi-supervised semantic segmentation on the Cityscapes dataset. Moreover, Superpixel-mix improves the reliability of semantic segmentation by reducing network uncertainty and bias, as confirmed by competitive results under strong distributions shift (adverse weather, image corruptions) and when facing out-of-distribution data.

Citations

• Strictly Proper Scoring Rules, Prediction, and Estimation
• THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1
• Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Examining the Impact of Blur on Recognition by Convolutional Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Improved Regularization of Convolutional Neural Networks with Cutout
• Semantic Foggy Scene Understanding with Synthetic Data
• Robust Physical-World Attacks on Deep Learning Models
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Evidential Deep Learning to Quantify Classification Uncertainty
• Manifold Mixup: Better Representations by Interpolating Hidden States
• DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation
• High-Resolution Representations for Labeling Pixels and Regions
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Ensembles: A Loss Landscape Perspective
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• Shortcut learning in deep neural networks
• Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup
• Robustness to Spurious Correlations via Human Annotations
• Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification
• Exploring Simple Siamese Representation Learning

Cited by

• HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation
• D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition

Related

• A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation
• Efficient semantic image segmentation with superpixel pooling
• Adaptive Superpixel for Active Learning in Semantic Segmentation
• Superpixel-based Semantic Segmentation Trained by Statistical Process Control
• Semantic Segmentation with Active Semi-Supervised Representation Learning