Bayes classifier cannot be learned from noisy responses with unknown noise rates

2023/04/13 by Soham Bakshi, Bakshi, Soham, Subha Maity +1
Computer Science · Engineering · Mathematics · #Advanced Statistical Methods and Models #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Water Systems and Optimization

paper · pdf · doi:10.48550/arxiv.2304.06574

openalex publication_date 2023/04/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Training a classifier with noisy labels typically requires the learner to specify the distribution of label noise, which is often unknown in practice. Although there have been some recent attempts to relax that requirement, we show that the Bayes decision rule is unidentified in most classification problems with noisy labels. This suggests it is generally not possible to bypass/relax the requirement. In the special cases in which the Bayes decision rule is identified, we develop a simple algorithm to learn the Bayes decision rule, that does not require knowledge of the noise distribution.

Related

• Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space
• Bayes and Naive Bayes Classifier
• Learning with Feature-Dependent Label Noise: A Progressive Approach
• Binary Classification with Instance and Label Dependent Label Noise
• Noise in Classification