Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning

2023/07/30 by Han, Zhi, Liu, Baichen, Lin, Shao-Bo +1
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2307.16203

Abstract

This paper studies the performance of deep convolutional neural networks (DCNNs) with zero-padding in feature extraction and learning. After verifying the roles of zero-padding in enabling translation-equivalence, and pooling in its translation-invariance driven nature, we show that with similar number of free parameters, any deep fully connected networks (DFCNs) can be represented by DCNNs with zero-padding. This demonstrates that DCNNs with zero-padding is essentially better than DFCNs in feature extraction. Consequently, we derive universal consistency of DCNNs with zero-padding and show its translation-invariance in the learning process. All our theoretical results are verified by numerical experiments including both toy simulations and real-data running.

Related

• Universal Consistency of Deep Convolutional Neural Networks
• Universal Approximation Property of Fully Convolutional Neural Networks with Zero Padding
• Universality of Deep Convolutional Neural Networks
• On the Behavior of Convolutional Nets for Feature Extraction
• Learning Ability of Interpolating Deep Convolutional Neural Networks