A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise

2020/03/15 by Mark Collier, Collier, Mark, Basil Mustafa +7 · 1 citation
Computer Science · #Machine Learning and Data Classification #Anomaly Detection Techniques and Applications #Adversarial Robustness in Machine Learning

paper · pdf · doi:10.48550/arxiv.2003.06778

Abstract

Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. We assume an underlying heteroscedastic generative process for noisy labels. To make gradient based training feasible we use a temperature parameterized softmax as a smooth approximation to the assumed generative process. We illustrate that the softmax temperature controls a bias-variance trade-off for the approximation. By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.

Citations

• Auto-Encoding Variational Bayes
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Are we done with ImageNet?

Cited by

• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model

Related

• Correlated Input-Dependent Label Noise in Large-Scale Image\n Classification
• Label Noise Types and Their Effects on Deep Learning
• Learning to Learn from Noisy Labeled Data
• Learning Deep Networks from Noisy Labels with Dropout Regularization
• Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks