Mathematics of Deep Learning

2017/12/13 by Rene Vidal, René Vidal, Vidal, Rene +6 · 1 voice · 3 citations
Computer Science · Engineering · #Neural Networks and Applications #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1712.04741

openalex publication_date 2017/12/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recently there has been a dramatic increase in the performance of recognition systems due to the introduction of deep architectures for representation learning and classification. However, the mathematical reasons for this success remain elusive. This tutorial will review recent work that aims to provide a mathematical justification for several properties of deep networks, such as global optimality, geometric stability, and invariance of the learned representations.

Citations

• The information bottleneck method
• The Loss Surfaces of Multilayer Networks
• Geometric Deep Learning: Going beyond Euclidean data
• Opening the Black Box of Deep Neural Networks via Information

Cited by

• Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex
• Algorithmic Regularization in Learning Deep Homogeneous Models: Layers\n are Automatically Balanced
• Non-attracting Regions of Local Minima in Deep and Wide Neural Networks

Discussions

• Mathematics of Deep Learning [pdf] [hn, 220 points, 38 comments]

Related

• The Modern Mathematics of Deep Learning
• Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
• Mathematical theory of deep learning
• A Structural Approach to the Design of Domain Specific Neural Network Architectures
• Deep Learning of Representations: Looking Forward