2021/02/23 by Sebastian Stabinger, Stabinger, Sebastian, David Peer +3
Computer Science · #Advanced Neural Network Applications #Machine Learning and Algorithms #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2102.11944
Convolutional neural networks have established themselves over the past years\nas the state of the art method for image classification, and for many datasets,\nthey even surpass humans in categorizing images. Unfortunately, the same\narchitectures perform much worse when they have to compare parts of an image to\neach other to correctly classify this image.\n Until now, no well-formed theoretical argument has been presented to explain\nthis deficiency. In this paper, we will argue that convolutional layers are of\nlittle use for such problems, since comparison tasks are global by nature, but\nconvolutional layers are local by design. We will use this insight to\nreformulate a comparison task into a sorting task and use findings on sorting\nnetworks to propose a lower bound for the number of parameters a neural network\nneeds to solve comparison tasks in a generalizable way. We will use this lower\nbound to argue that attention, as well as iterative/recurrent processing, is\nneeded to prevent a combinatorial explosion.\n