2022/11/02 by Carol Mak, Mak, Carol, Fabian Zaiser +3 · 1 citation
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Bayesian Modeling and Causal Inference #Bayesian probability #Computation (stat.CO) #Computer science #Correctness #Econometrics #FOS: Computer and information sciences #Formal Methods in Verification #Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine learning #Markov chain #Markov chain Monte Carlo #Mathematical optimization #Mathematics #Monte Carlo method #Nonparametric statistics #Probabilistic logic #Programming Languages (cs.PL) #Statistics #Theoretical computer science
paper · pdf · doi:10.48550/arxiv.2211.01100
openalex publication_date 2022/11/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
A challenging problem in probabilistic programming is to develop inference algorithms that work for arbitrary programs in a universal probabilistic programming language (PPL). We present the nonparametric involutive Markov chain Monte Carlo (NP-iMCMC) algorithm as a method for constructing MCMC inference algorithms for nonparametric models expressible in universal PPLs. Building on the unifying involutive MCMC framework, and by providing a general procedure for driving state movement between dimensions, we show that NP-iMCMC can generalise numerous existing iMCMC algorithms to work on nonparametric models. We prove the correctness of the NP-iMCMC sampler. Our empirical study shows that the existing strengths of several iMCMC algorithms carry over to their nonparametric extensions. Applying our method to the recently proposed Nonparametric HMC, an instance of (Multiple Step) NP-iMCMC, we have constructed several nonparametric extensions (all of which new) that exhibit significant performance improvements.