Signal Recovery from Pooling Representations

2013/11/16 by Joan Bruna, Arthur Szlam, Bruna, Joan +3 · 3 citations
Computer Science · Engineering · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #Image and Signal Denoising Methods #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Sparse and Compressive Sensing Techniques #stat.ML

paper · pdf · doi:10.48550/arxiv.1311.4025

17 pages, 3 figures

openalex publication_date 2013/11/16 · arxiv created 2014/02/27 · arxiv updated 2014/03/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work we compute lower Lipschitz bounds of ℓ_p pooling operators for p=1, 2, ∞ as well as ℓ_p pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.

Citations

• Invertibility and Robustness of Phaseless Reconstruction
• rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation

Cited by

• The Sparse Manifold Transform
• Globally Injective ReLU Networks
• Data-driven Regularization via Racecar Training for Generalizing Neural Networks

Related

• Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval
• Pooling Revisited: Your Receptive Field is Suboptimal
• A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
• LiftPool: Bidirectional ConvNet Pooling
• Detail-Preserving Pooling in Deep Networks