Efficient Image Dataset Classification Difficulty Estimation for Predicting Deep-Learning Accuracy

2018/03/26 by Scheidegger, Florian, Istrate, Roxana, Mariani, Giovanni +3 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1803.09588

Abstract

In the deep-learning community new algorithms are published at an incredible pace. Therefore, solving an image classification problem for new datasets becomes a challenging task, as it requires to re-evaluate published algorithms and their different configurations in order to find a close to optimal classifier. To facilitate this process, before biasing our decision towards a class of neural networks or running an expensive search over the network space, we propose to estimate the classification difficulty of the dataset. Our method computes a single number that characterizes the dataset difficulty 27x faster than training state-of-the-art networks. The proposed method can be used in combination with network topology and hyper-parameter search optimizers to efficiently drive the search towards promising neural-network configurations.

Cited by

• Dataset-Adaptive Dimensionality Reduction

Related

• Data-Efficient Deep Learning Method for Image Classification Using Data Augmentation, Focal Cosine Loss, and Ensemble
• Problem-dependent attention and effort in neural networks with applications to image resolution and model selection
• "You might also like this model": Data Driven Approach for Recommending Deep Learning Models for Unknown Image Datasets
• Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights
• Computational issues in Optimization for Deep networks