Deep Neural Network Structures Solving Variational Inequalities

2018/08/22 by Combettes, Patrick L., Pesquet, Jean-Christophe · 4 citations
#FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.1808.07526

Abstract

Motivated by structures that appear in deep neural networks, we investigate nonlinear composite models alternating proximity and affine operators defined on different spaces. We first show that a wide range of activation operators used in neural networks are actually proximity operators. We then establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties. It is shown that the limit of the resulting process solves a variational inequality which, in general, does not derive from a minimization problem.

Cited by

• A golden ratio self-adaptive algorithm for solving a quasimonotone variational inequality in banach spaces
• Exact and Approximate Solvability of Systems Involving Proximity Operators in Hilbert Spaces
• Circuit realization and hardware linearization of monotone operator equilibrium networks
• Plug and Play Splitting Techniques for Poisson Image Restoration

Related

• Approximation Rates for Neural Networks with Encodable Weights in\n Smoothness Spaces
• New Approximation Results and Optimal Estimation for Fully Connected Deep Neural Networks
• A Numerical Investigation of the Minimum Width of a Neural Network
• Neural networks: deep, shallow, or in between?
• Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation