Batch learning equals online learning in Bayesian supervised learning

2025/10/19 by Hông Vân Lê, Lê, Hông Vân
Computer Science · #Bayesian Methods and Mixture Models #Gaussian Processes and Bayesian Inference #Machine Learning and Algorithms #math.ST #stat.TH

paper · pdf · doi:10.48550/arxiv.2510.16892

openalex publication_date 2025/10/19 · openalex created_date 2025/10/22 · openalex updated_date 2026/08/01

Abstract

In this paper we study Bayesian supervised learning models proposed by Lê in \citeLe2025. Using functoriality of probabilistic morphisms, we prove that sequential and batch Bayesian inversions coincide in supervised learning models with conditionally independent (possibly non-i.i.d.) data \citeLe2025. This equivalence holds without domination or discreteness assumptions on sampling operators. We derive a recursive formula for posterior predictive distributions, which reduces to the Kalman filter in Gaussian process regression. For Souslin label spaces Y and arbitrary input sets X, we characterize probability measures on P(Y)^X via projective systems, generalizing Orbanz \citeOrbanz2011. We revisit MacEachern's Dependent Dirichlet Processes (DDP) \citeMacEachern2000 using copula-based constructions \citeBJQ2012 and show how to compute posterior predictive distributions in universal Bayesian supervised models with DDP priors.

Citations

• A probabilistic view on predictive constructions for Bayesian learning
• Analyticity and sparsity in uncertainty quantification for PDEs with Gaussian random field inputs

Related

• Probabilistic morphisms and Bayesian supervised learning
• A category theory framework for Bayesian learning
• Bayesian machine learning via category theory
• Online Bernstein-von Mises theorem
• Online Bernstein-von Mises theorem