Hessian-informed Hamiltonian Monte Carlo for high-dimensional problems

2023/03/28 by Mina Karimi, Kaushik Dayal, Karimi, Mina +3
Computer Science · Mathematics · #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Markov Chains and Monte Carlo Methods #Probability (math.PR) #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2305.01576

openalex publication_date 2023/03/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We investigate the effect of using local and non-local second derivative information on the performance of Hamiltonian Monte Carlo (HMC) sampling methods, for high-dimension non-Gaussian distributions, with application to Bayesian inference and nonlinear inverse problems. The Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) method uses second and third derivative information to improve the performance of the HMC approach. We propose using the local Hessian information at the start of each iteration, instead of re-calculating the higher order derivatives in all sub-steps of the leapfrog updating algorithm. We compare the result of Hessian-informed HMC method using the local and nonlocal Hessian information, in a test bed of a high-dimensional log-normal distribution, related to a problem of inferring soil properties.

Related

• Riemannian Manifold Hamiltonian Monte Carlo
• Modified Cholesky Riemann Manifold Hamiltonian Monte Carlo: Exploiting\n Sparsity for Fast Sampling of High-dimensional Targets
• Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models
• Adaptive Riemannian Manifold Hamiltonian Monte Carlo with Hierarchical Metric
• On Numerical Considerations for Riemannian Manifold Hamiltonian Monte Carlo