Consistent Relative Confidence and Label-Free Model Selection for Convolutional Neural Networks

2021/08/26 by Bin Liu, Liu, Bin · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2108.11845

openalex publication_date 2021/08/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we are concerned with image classification with deep convolutional neural networks (CNNs). We focus on the following question: given a set of candidate CNN models, how to select the right one with the best generalization property for the current task? Current model selection methods all require access to a batch of labeled data for computing a pre-specified performance metric, such as the cross-entropy loss, the classification error rate and the negative log-likelihood. In many practical cases, labels are not available in time as labeling itself is a time-consuming and expensive task. To this end, we propose an approach to CNN model selection using only unlabeled data. We develop this method based on a principle termed consistent relative confidence. Experimental results on benchmark datasets demonstrate the effectiveness and efficiency of our method.

Cited by

• Can We Recycle Our Old Models? An Empirical Evaluation of Model Selection Mechanisms for AIOps Solutions

Related

• Understanding deep learning (still) requires rethinking generalization
• Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers
• Understanding deep learning requires rethinking generalization
• Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks
• Best Practices for Convolutional Neural Networks Applied to Object Recognition in Images