Joint Negative and Positive Learning for Noisy Labels

2021/04/14 by Young‐Dong Kim, Juseung Yun, Kim, Youngdong +5 · 3 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2104.06574

openalex publication_date 2021/04/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Training of Convolutional Neural Networks (CNNs) with data with noisy labels is known to be a challenge. Based on the fact that directly providing the label to the data (Positive Learning; PL) has a risk of allowing CNNs to memorize the contaminated labels for the case of noisy data, the indirect learning approach that uses complementary labels (Negative Learning for Noisy Labels; NLNL) has proven to be highly effective in preventing overfitting to noisy data as it reduces the risk of providing faulty target. NLNL further employs a three-stage pipeline to improve convergence. As a result, filtering noisy data through the NLNL pipeline is cumbersome, increasing the training cost. In this study, we propose a novel improvement of NLNL, named Joint Negative and Positive Learning (JNPL), that unifies the filtering pipeline into a single stage. JNPL trains CNN via two losses, NL+ and PL+, which are improved upon NL and PL loss functions, respectively. We analyze the fundamental issue of NL loss function and develop new NL+ loss function producing gradient that enhances the convergence of noisy data. Furthermore, PL+ loss function is designed to enable faster convergence to expected-to-be-clean data. We show that the NL+ and PL+ train CNN simultaneously, significantly simplifying the pipeline, allowing greater ease of practical use compared to NLNL. With a simple semi-supervised training technique, our method achieves state-of-the-art accuracy for noisy data classification based on the superior filtering ability.

Citations

• CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise
• Learning to Reweight Examples for Robust Deep Learning
• Dimensionality-Driven Learning with Noisy Labels
• How does Disagreement Help Generalization against Label Corruption?
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Unlearning the Noisy Correspondence Makes CLIP More Robust
• FixCLR: Negative-Class Contrastive Learning for Semi-Supervised Domain Generalization
• Think Twice before Adaptation: Improving Adaptability of DeepFake Detection via Online Test-Time Adaptation

Related

• NLNL: Negative Learning for Noisy Labels
• A Novel Perspective for Positive-Unlabeled Learning via Noisy Labels
• Noise-Robust Bidirectional Learning with Dynamic Sample Reweighting
• Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning
• Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction