Supervised learning with probabilistic morphisms and kernel mean embeddings

2023/05/10 by Hông Vân Lê, Lê, Hông Vân · 1 citation
Computer Science · #18N99 #46N30 #60B10 #62G05 #Bayesian Methods and Mixture Models #Category Theory (math.CT) #FOS: Computer and information sciences #FOS: Mathematics #Functional Analysis (math.FA) #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Neural Networks and Applications #Probability (math.PR) #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2305.06348

openalex publication_date 2023/05/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper I propose a generative model of supervised learning that unifies two approaches to supervised learning, using a concept of a correct loss function. Addressing two measurability problems, which have been ignored in statistical learning theory, I propose to use convergence in outer probability to characterize the consistency of a learning algorithm. Building upon these results, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik-Stefanuyk's regularization method for solving stochastic ill-posed problems, and using it to prove the generalizability of overparameterized supervised learning models.

Cited by

• Categorical and geometric methods in statistical, manifold, and machine learning

Related

• Probabilistic morphisms and Bayesian supervised learning
• Computing the Shattering Coefficient of Supervised Learning Algorithms
• Trained Model in Supervised Deep Learning is a Conditional Risk Minimizer
• Loss factorization, weakly supervised learning and label noise robustness
• Iterative Regularization for Learning with Convex Loss Functions