2020/03/15 by Mark Collier, Collier, Mark, Basil Mustafa +7 · 2 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Machine Learning and Data Classification #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2003.06778
arxiv created 2020/11/12 · arxiv updated 2020/11/16
Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. We assume an underlying heteroscedastic generative process for noisy labels. To make gradient based training feasible we use a temperature parameterized softmax as a smooth approximation to the assumed generative process. We illustrate that the softmax temperature controls a bias-variance trade-off for the approximation. By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.