Towards a mathematical understanding of learning from few examples with nonlinear feature maps

2022/11/07 by Oliver J. Sutton, Sutton, Oliver J., Alexander N. Gorban +3
Computer Science · #68Q32 #68T05 #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Face and Expression Recognition #Image Retrieval and Classification Techniques #Machine Learning (cs.LG) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2211.03607

openalex publication_date 2022/11/07 · openalex created_date 2022/11/13 · openalex updated_date 2026/07/28

Abstract

We consider the problem of data classification where the training set consists of just a few data points. We explore this phenomenon mathematically and reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities. The main thrust of our analysis is to reveal the influence on the model's generalisation capabilities of nonlinear feature transformations mapping the original data into high, and possibly infinite, dimensional spaces.

Related

• Learning from few examples with nonlinear feature maps
• Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
• A study of the classification of low-dimensional data with supervised\n manifold learning
• From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning
• Continuous Learning: Engineering Super Features With Feature Algebras