2015/12/19 by Thomas Wiatowski, Helmut Bölcskei, Wiatowski, Thomas +1 · 1 voice · 2 citations
Computer Science · Earth and Planetary Sciences · Physics and Astronomy · #Image and Signal Denoising Methods #Seismic Imaging and Inversion Techniques #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.1512.06293
Deep convolutional neural networks have led to breakthrough results in\nnumerous practical machine learning tasks such as classification of images in\nthe ImageNet data set, control-policy-learning to play Atari games or the board\ngame Go, and image captioning. Many of these applications first perform feature\nextraction and then feed the results thereof into a trainable classifier. The\nmathematical analysis of deep convolutional neural networks for feature\nextraction was initiated by Mallat, 2012. Specifically, Mallat considered\nso-called scattering networks based on a wavelet transform followed by the\nmodulus non-linearity in each network layer, and proved translation invariance\n(asymptotically in the wavelet scale parameter) and deformation stability of\nthe corresponding feature extractor. This paper complements Mallat's results by\ndeveloping a theory that encompasses general convolutional transforms, or in\nmore technical parlance, general semi-discrete frames (including\nWeyl-Heisenberg filters, curvelets, shearlets, ridgelets, wavelets, and learned\nfilters), general Lipschitz-continuous non-linearities (e.g., rectified linear\nunits, shifted logistic sigmoids, hyperbolic tangents, and modulus functions),\nand general Lipschitz-continuous pooling operators emulating, e.g.,\nsub-sampling and averaging. In addition, all of these elements can be different\nin different network layers. For the resulting feature extractor we prove a\ntranslation invariance result of vertical nature in the sense of the features\nbecoming progressively more translation-invariant with increasing network\ndepth, and we establish deformation sensitivity bounds that apply to signal\nclasses such as, e.g., band-limited functions, cartoon functions, and Lipschitz\nfunctions.\n