2020/06/08 by Pravendra Singh, Singh, Pravendra, Pratik Mazumder +3
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.2006.04406
This work presents a novel training technique for deep neural networks that\nmakes use of additional data from a distribution that is different from that of\nthe original input data. This technique aims to reduce overfitting and improve\nthe generalization performance of the network. Our proposed technique, namely\nPassive Batch Injection Training Technique (PBITT), even reduces the level of\noverfitting in networks that already use the standard techniques for reducing\noverfitting such as L2 regularization and batch normalization, resulting in\nsignificant accuracy improvements. Passive Batch Injection Training Technique\n(PBITT) introduces a few passive mini-batches into the training process that\ncontain data from a distribution that is different from the input data\ndistribution. This technique does not increase the number of parameters in the\nfinal model and also does not increase the inference (test) time but still\nimproves the performance of deep CNNs. To the best of our knowledge, this is\nthe first work that makes use of different data distribution to aid the\ntraining of convolutional neural networks (CNNs). We thoroughly evaluate the\nproposed approach on standard architectures: VGG, ResNet, and WideResNet, and\non several popular datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. We\nobserve consistent accuracy improvement by using the proposed technique. We\nalso show experimentally that the model trained by our technique generalizes\nwell to other tasks such as object detection on the MS-COCO dataset using\nFaster R-CNN. We present extensive ablations to validate the proposed approach.\nOur approach improves the accuracy of VGG-16 by a significant margin of 2.1%\nover the CIFAR-100 dataset.\n