2023/08/14 by Jungsoo Lee, Debasmit Das, Lee, Jungsoo +5 · 10 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Crowds #Domain Adaptation and Few-Shot Learning #Entropy (arrow of time) #FOS: Computer and information sciences #Image and Signal Denoising Methods #Low Confidence #Machine learning #Mathematics #Minification #Pattern recognition (psychology) #Speech and Audio Processing #Statistics
paper · pdf · doi:10.48550/arxiv.2308.06879
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/08/14 · openalex created_date 2023/08/16 · openalex updated_date 2026/07/28
Test-time adaptation (TTA) methods, which generally rely on the model's predictions (e.g., entropy minimization) to adapt the source pretrained model to the unlabeled target domain, suffer from noisy signals originating from 1) incorrect or 2) open-set predictions. Long-term stable adaptation is hampered by such noisy signals, so training models without such error accumulation is crucial for practical TTA. To address these issues, including open-set TTA, we propose a simple yet effective sample selection method inspired by the following crucial empirical finding. While entropy minimization compels the model to increase the probability of its predicted label (i.e., confidence values), we found that noisy samples rather show decreased confidence values. To be more specific, entropy minimization attempts to raise the confidence values of an individual sample's prediction, but individual confidence values may rise or fall due to the influence of signals from numerous other predictions (i.e., wisdom of crowds). Due to this fact, noisy signals misaligned with such 'wisdom of crowds', generally found in the correct signals, fail to raise the individual confidence values of wrong samples, despite attempts to increase them. Based on such findings, we filter out the samples whose confidence values are lower in the adapted model than in the original model, as they are likely to be noisy. Our method is widely applicable to existing TTA methods and improves their long-term adaptation performance in both image classification (e.g., 49.4% reduced error rates with TENT) and semantic segmentation (e.g., 11.7% gain in mIoU with TENT).