Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels

2018/04/18 by Bo Han, Quanming Yao, Han, Bo +13 · 61 citations
Computer Science · #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1804.06872

openalex publication_date 2018/04/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep learning with noisy labels is practically challenging, as the capacity of deep models is so high that they can totally memorize these noisy labels sooner or later during training. Nonetheless, recent studies on the memorization effects of deep neural networks show that they would first memorize training data of clean labels and then those of noisy labels. Therefore in this paper, we propose a new deep learning paradigm called Co-teaching for combating with noisy labels. Namely, we train two deep neural networks simultaneously, and let them teach each other given every mini-batch: firstly, each network feeds forward all data and selects some data of possibly clean labels; secondly, two networks communicate with each other what data in this mini-batch should be used for training; finally, each network back propagates the data selected by its peer network and updates itself. Empirical results on noisy versions of MNIST, CIFAR-10 and CIFAR-100 demonstrate that Co-teaching is much superior to the state-of-the-art methods in the robustness of trained deep models.

Citations

• A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
• Noise-tolerant learning, the parity problem, and the statistical query model
• Positive-Unlabeled Learning with Non-Negative Risk Estimator
• A Closer Look at Memorization in Deep Networks
• Learning from Noisy Labels with Distillation

Cited by

• XMIX: Combating Extremely Noisy Labels via Local Smoothness in Self-Supervised Feature Space
• Multiclass Classification without Labels via Posterior Simplex Geometry
• SELECT: Detecting Label Errors in Real-world Scene Text Data
• Adjudicator: Correcting Noisy Labels with a KG-Informed Council of LLM Agents
• Weak-to-Strong Generalization Enables Fully Automated De Novo Training of Multi-head Mask-RCNN Model for Segmenting Densely Overlapping Cell Nuclei in Multiplex Whole-slice Brain Images
• Benchmarking Real-World Medical Image Classification with Noisy Labels: Challenges, Practice, and Outlook
• MANTRA: a Framework for Multi-stage Adaptive Noise TReAtment During Training
• Lost in Time? A Meta-Learning Framework for Time-Shift-Tolerant Physiological Signal Transformation
• Anomaly Detection with Adaptive and Aggressive Rejection for Contaminated Training Data
• FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning
• Learning to Clean: Reinforcement Learning for Noisy Label Correction
• Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic
• Auxiliary Gene Learning: Spatial Gene Expression Estimation by Auxiliary Gene Selection
• SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data
• Step-E: A Differentiable Data Cleaning Framework for Robust Learning with Noisy Labels
• Online Data Curation for Object Detection via Marginal Contributions to Dataset-level Average Precision
• LogPurge: Log Data Purification for Anomaly Detection via Rule-Enhanced Filtering
• Zero-Shot Grammar Competency Estimation Using Large Language Model Generated Pseudo Labels
• DK-Root: A Joint Data-and-Knowledge-Driven Framework for Root Cause Analysis of QoE Degradations in Mobile Networks
• HiLoMix: Robust High- and Low-Frequency Graph Learning Framework for Mixing Address Association
• LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mapping
• HEDN: A Hard-Easy Dual Network with Source Reliability Assessment for Cross-Subject EEG Emotion Recognition
• Local K-Similarity Constraint for Federated Learning with Label Noise
• Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
• Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration
• Benchmarking noisy label detection methods
• Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees
• EReLiFM: Evidential Reliability-Aware Residual Flow Meta-Learning for Open-Set Domain Generalization under Noisy Labels
• Robust Learning of Diffusion Models with Extremely Noisy Conditions
• Weed Out, Then Harvest: Dual Low-Rank Adaptation is an Effective Noisy Label Detector for Noise-Robust Learning
• Exploring Single Domain Generalization of LiDAR-based Semantic Segmentation under Imperfect Labels
• Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data
• Rethinking Consistent Multi-Label Classification Under Inexact Supervision
• Learning Robust Diffusion Models from Imprecise Supervision
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
• LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training
• Segment-to-Act: Label-Noise-Robust Action-Prompted Video Segmentation Towards Embodied Intelligence
• LLM-Guided Co-Training for Text Classification
• Autoguided Online Data Curation for Diffusion Model Training
• Label-Noise Robust Generative Adversarial Networks
• SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing
• Similarity-based Outlier Detection for Noisy Object Re-Identification Using Beta Mixtures
• Backdoor Attacks and Defenses in Computer Vision Domain: A Survey
• Tackling the Noisy Elephant in the Room: Label Noise-robust Out-of-Distribution Detection via Loss Correction and Low-rank Decomposition
• Parameter-Free Logit Distillation via Sorting Mechanism
• From Noisy Labels to Intrinsic Structure: A Geometric-Structural Dual-Guided Framework for Noise-Robust Medical Image Segmentation
• Ordinal Adaptive Correction: A Data-Centric Approach to Ordinal Image Classification with Noisy Labels
• FedThief: Harming Others to Benefit Oneself in Self-Centered Federated Learning
• Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction
• A Two-Stream Mutual Attention Network for Semi-supervised Biomedical Segmentation with Noisy Labels
• From Entity Reliability to Clean Feedback: An Entity-Aware Denoising Framework Beyond Interaction-Level Signals
• Collaborative Learning of Scattering and Deep Features for SAR Target Recognition with Noisy Labels
• Detecting Mislabeled and Corrupted Data via Pointwise Mutual Information
• Prototype-Guided Curriculum Learning for Zero-Shot Learning
• Introducing Fractional Classification Loss for Robust Learning with Noisy Labels
• MCA: 2D-3D Retrieval with Noisy Labels via Multi-level Adaptive Correction and Alignment
• Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labels
• ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise
• Nested Graph Pseudo-Label Refinement for Noisy Label Domain Adaptation Learning
• Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models
• Robust Classification under Noisy Labels: A Geometry-Aware Reliability Framework for Foundation Models

Related

• How does Disagreement Help Generalization against Label Corruption?
• Combating noisy labels by agreement: A joint training method with co-regularization
• CrossSplit: Mitigating Label Noise Memorization through Data Splitting
• Noisy Concurrent Training for Efficient Learning under Label Noise
• Collaborative Learning for Deep Neural Networks