Predicting Generalization in Deep Learning via Local Measures of Distortion

2020/12/13 by Rajagopal, Abhejit, Madala, Vamshi C., Chandrasekaran, Shivkumar +1
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2012.06969

Abstract

We study generalization in deep learning by appealing to complexity measures originally developed in approximation and information theory. While these concepts are challenged by the high-dimensional and data-defined nature of deep learning, we show that simple vector quantization approaches such as PCA, GMMs, and SVMs capture their spirit when applied layer-wise to deep extracted features giving rise to relatively inexpensive complexity measures that correlate well with generalization performance. We discuss our results in 2020 NeurIPS PGDL challenge.

Related

• NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
• Representation Based Complexity Measures for Predicting Generalization in Deep Learning
• Predicting Generalization in Deep Learning via Metric Learning -- PGDL\n Shared task
• Fantastic Generalization Measures and Where to Find Them
• In Search of Robust Measures of Generalization