vix.ing · top · new · best · stats · spec

Revealing the Underlying Patterns: Investigating Dataset Similarity, Performance, and Generalization

2023/08/07 by Akshit Achara, Achara, Akshit, Ram Krishna Pandey +1
Computer Science · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2308.03580

openalex publication_date 2023/08/07 · openalex created_date 2023/08/09 · openalex updated_date 2026/07/28

Abstract

Supervised deep learning models require significant amount of labeled data to achieve an acceptable performance on a specific task. However, when tested on unseen data, the models may not perform well. Therefore, the models need to be trained with additional and varying labeled data to improve the generalization. In this work, our goal is to understand the models, their performance and generalization. We establish image-image, dataset-dataset, and image-dataset distances to gain insights into the model's behavior. Our proposed distance metric when combined with model performance can help in selecting an appropriate model/architecture from a pool of candidate architectures. We have shown that the generalization of these models can be improved by only adding a small number of unseen images (say 1, 3 or 7) into the training set. Our proposed approach reduces training and annotation costs while providing an estimate of model performance on unseen data in dynamic environments.

Related