2021/11/29 by Lorenzo Brigato, Brigato, Lorenzo, Luca Iocchi +1
Computer Science · Medicine · #Advanced Neural Network Applications #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2111.14493
openalex publication_date 2021/11/29 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Deep neural networks represent the gold standard for image classification. However, they usually need large amounts of data to reach superior performance. In this work, we focus on image classification problems with a few labeled examples per class and improve data efficiency by using an ensemble of relatively small networks. For the first time, our work broadly studies the existing concept of neural ensembling in domains with small data, through extensive validation using popular datasets and architectures. We compare ensembles of networks to their deeper or wider single competitors given a total fixed computational budget. We show that ensembling relatively shallow networks is a simple yet effective technique that is generally better than current state-of-the-art approaches for learning from small datasets. Finally, we present our interpretation according to which neural ensembles are more sample efficient because they learn simpler functions.