Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

2013/01/16 by Zeiler, Matthew D., Fergus, Rob · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1301.3557

Abstract

We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined with other regularization approaches, such as dropout and data augmentation. We achieve state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.

Cited by

• AMD-HookNet++: Evolution of AMD-HookNet with Hybrid CNN-Transformer Feature Enhancement for Glacier Calving Front Segmentation
• OS-HGAdapter: Open Semantic Hypergraph Adapter for Large Language Models Assisted Entropy-Enhanced Image-Text Alignment

Related

• Stochastic Subsampling With Average Pooling
• Max-Pooling Dropout for Regularization of Convolutional Neural Networks
• Regularizing Neural Networks via Stochastic Branch Layers
• S3Pool: Pooling with Stochastic Spatial Sampling
• Pooling Methods in Deep Neural Networks, a Review