Probabilistic End-to-end Noise Correction for Learning with Noisy Labels

2019/03/19 by Kun Yi, Jianxin Wu, Yi, Kun +1 · 67 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial intelligence #Artificial neural network #Computer science #Data mining #End-to-end principle #Engineering #Machine Learning and Algorithms #Machine Learning and Data Classification #Machine learning #Noise (video) #Overfitting #Pattern recognition (psychology) #Pencil (optics) #Probabilistic logic #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1903.07788

published in arXiv (Cornell University) (Cornell University) · CVPR 2019

arxiv created 2019/03/19 · openalex publication_date 2019/03/19 · arxiv updated 2019/03/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Deep learning has achieved excellent performance in various computer vision tasks, but requires a lot of training examples with clean labels. It is easy to collect a dataset with noisy labels, but such noise makes networks overfit seriously and accuracies drop dramatically. To address this problem, we propose an end-to-end framework called PENCIL, which can update both network parameters and label estimations as label distributions. PENCIL is independent of the backbone network structure and does not need an auxiliary clean dataset or prior information about noise, thus it is more general and robust than existing methods and is easy to apply. PENCIL outperforms previous state-of-the-art methods by large margins on both synthetic and real-world datasets with different noise types and noise rates. Experiments show that PENCIL is robust on clean datasets, too.

Cited by

• Robustness of Probabilistic Models to Low-Quality Data: A Multi-Perspective Analysis
• DenoGrad: A Gradient-Based Framework for Data Refinement in Tabular and Time-Series Learning
• Combating noisy labels by agreement: A joint training method with co-regularization
• Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees
• Learning Robust Diffusion Models from Imprecise Supervision
• Curvature Learning for Generalization of Hyperbolic Neural Networks
• MoPro: Webly Supervised Learning with Momentum Prototypes
• A Tale Of Two Long Tails
• Label Noise Types and Their Effects on Deep Learning
• COLUR: Confidence-Oriented Learning, Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinement
• Semi-supervised Learning for Few-shot Image-to-Image Translation
• Why Can Accurate Models Be Learned from Inaccurate Annotations?
• Reliable Label Bootstrapping for Semi-Supervised Learning
• Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
• Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback
• Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data
• PENCIL: Deep Learning with Noisy Labels
• Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression

Related

• PENCIL: Deep Learning with Noisy Labels
• Synergistic Network Learning and Label Correction for Noise-robust Image Classification
• Learning to Learn from Noisy Labeled Data
• Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks
• Noisy Label Processing for Classification: A Survey