2024/10/02 by Schyan Zafar, Zafar, Schyan, Geoff K. Nicholls +1 · 1 citation
Computer Science · Engineering · #Applications (stat.AP) #Control Systems and Identification #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning and Algorithms #Methodology (stat.ME)
paper · pdf · doi:10.48550/arxiv.2410.01475
openalex publication_date 2024/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Generalised Bayesian Inference (GBI) attempts to address model misspecification in a standard Bayesian setup by tempering the likelihood. The likelihood is raised to a fractional power, called the learning rate, which reduces its importance in the posterior and has been established as a method to address certain kinds of model misspecification. Posterior Predictive Checks (PPC) attempt to detect model misspecification by locating a diagnostic, computed on the observed data, within the posterior predictive distribution of the diagnostic. This can be used to construct a hypothesis test where a small p-value indicates potential misfit. The recent Embedded Diachronic Sense Change (EDiSC) model suffers from misspecification and benefits from likelihood tempering. Using EDiSC as a case study, this exploratory work examines whether PPC could be used in a novel way to set the learning rate in a GBI setup. Specifically, the learning rate selected is the lowest value for which a hypothesis test using the log likelihood diagnostic is not rejected at the 10% level. The experimental results are promising, though not definitive, and indicate the need for further research along the lines suggested here.