2023/11/27 by Shikai Qiu, Tim G. J. Rudner, Qiu, Shikai +5 · 3 citations
Computer Science · Engineering · Mathematics · Medicine · #A priori and a posteriori #AI in cancer detection #Algorithm #Applied mathematics #Artificial intelligence #Artificial neural network #Bayesian probability #Computer science #Estimation theory #FOS: Computer and information sciences #Function (biology) #Generalization #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical analysis #Mathematical optimization #Mathematics #Maxima and minima #Maximum a posteriori estimation #Maximum likelihood #Medical Imaging and Analysis #Overfitting #Parameter space #Posterior probability #Radiomics and Machine Learning in Medical Imaging #Robustness (evolution) #Statistics
paper · pdf · doi:10.48550/arxiv.2311.15990
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/11/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Standard regularized training procedures correspond to maximizing a posterior distribution over parameters, known as maximum a posteriori (MAP) estimation. However, model parameters are of interest only insomuch as they combine with the functional form of a model to provide a function that can make good predictions. Moreover, the most likely parameters under the parameter posterior do not generally correspond to the most likely function induced by the parameter posterior. In fact, we can re-parametrize a model such that any setting of parameters can maximize the parameter posterior. As an alternative, we investigate the benefits and drawbacks of directly estimating the most likely function implied by the model and the data. We show that this procedure leads to pathological solutions when using neural networks and prove conditions under which the procedure is well-behaved, as well as a scalable approximation. Under these conditions, we find that function-space MAP estimation can lead to flatter minima, better generalization, and improved robustness to overfitting.