Training on test data: Removing near duplicates in Fashion-MNIST

2019/06/19 by Christopher Geier, Geier, Christopher
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI)

paper · pdf · doi:10.48550/arxiv.1906.08255

Abstract

MNIST and Fashion MNIST are extremely popular for testing in the machine learning space. Fashion MNIST improves on MNIST by introducing a harder problem, increasing the diversity of testing sets, and more accurately representing a modern computer vision task. In order to increase the data quality of FashionMNIST, this paper investigates near duplicate images between training and testing sets. Near-duplicates between testing and training sets artificially increase the testing accuracy of machine learning models. This paper identifies near-duplicate images in Fashion MNIST and proposes a dataset with near-duplicates removed.

Related

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shifts
• Do We Train on Test Data? Purging CIFAR of Near-Duplicates
• Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing
• PADS: Policy-Adapted Sampling for Visual Similarity Learning