Learning finite-dimensional coding schemes with nonlinear reconstruction\n maps

2018/12/23 by Jae-Ho Lee, Maxim Raginsky, Lee, Jaeho +1 · 1 citation
Mathematics · Computer Science · #Statistical Methods and Inference #Markov Chains and Monte Carlo Methods #Topological and Geometric Data Analysis

paper · pdf · doi:10.48550/arxiv.1812.09658

Abstract

This paper generalizes the Maurer--Pontil framework of finite-dimensional\nlossy coding schemes to the setting where a high-dimensional random vector is\nmapped to an element of a compact set of latent representations in a\nlower-dimensional Euclidean space, and the reconstruction map belongs to a\ngiven class of nonlinear maps. Under this setup, which encompasses a broad\nclass of unsupervised representation learning problems, we establish a\nconnection to approximate generative modeling under structural constraints\nusing the tools from the theory of optimal transportation. Next, we consider\nproblem of learning a coding scheme on the basis of a finite collection of\ntraining samples and present generalization bounds that hold with high\nprobability. We then illustrate the general theory in the setting where the\nreconstruction maps are implemented by deep neural nets.\n

Cited by

• LSTM-PINN: An Hybrid Method for Prediction of Steady-State Electrohydrodynamic Flow

Related

• Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics
• A unified framework for learning with nonlinear model classes from arbitrary linear samples
• Statistical Learning Theory for Neural Operators
• Dimensionality-Dependent Generalization Bounds for k-Dimensional Coding Schemes
• A solvable model of learning generative diffusion: theory and insights