2020/01/01 by Coenraad Mouton, Johannes C. Myburgh, Marelie H. Davel
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Convolutional neural network #Generative Adversarial Networks and Image Synthesis #Invariant (physics) #Kernel (algebra) #Pooling #STRIDE #Translation (biology) #cs.LG
paper · pdf · doi:10.1007/978-3-030-66151-9_17
published as Communications in Computer and Information Science (LNCS sub-series CCIS), vol 1342, pp 267-281, 2020
openalex publication_date 2020/01/01 · openalex created_date 2021/01/05 · arxiv created 2021/03/18 · arxiv updated 2021/03/19 · openalex updated_date 2026/08/05
Convolutional Neural Networks have become the standard for image classification tasks, however, these architectures are not invariant to translations of the input image. This lack of invariance is attributed to the use of stride which ignores the sampling theorem, and fully connected layers which lack spatial reasoning. We show that stride can greatly benefit translation invariance given that it is combined with sufficient similarity between neighbouring pixels, a characteristic which we refer to as local homogeneity. We also observe that this characteristic is dataset-specific and dictates the relationship between pooling kernel size and stride required for translation invariance. Furthermore we find that a trade-off exists between generalization and translation invariance in the case of pooling kernel size, as larger kernel sizes lead to better invariance but poorer generalization. Finally we explore the efficacy of other solutions proposed, namely global average pooling, anti-aliasing, and data augmentation, both empirically and through the lens of local homogeneity.