Tuning diagonal scale matrices for HMC

2024/03/12 by Jimmy Huy Tran, Tran, Jimmy Huy, Tore Selland Kleppe +1 · 3 citations
Computer Science · #Computation (stat.CO) #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (stat.ML) #Methodology (stat.ME)

paper · pdf · doi:10.48550/arxiv.2403.07495

openalex publication_date 2024/03/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Three approaches for adaptively tuning diagonal scale matrices for HMC are discussed and compared. The common practice of scaling according to estimated marginal standard deviations is taken as a benchmark. Scaling according to the mean log-target gradient (ISG), and a scaling method targeting that the frequency of when the underlying Hamiltonian dynamics crosses the respective medians should be uniform across dimensions, are taken as alternatives. Numerical studies suggest that the ISG method leads in many cases to more efficient sampling than the benchmark, in particular in cases with strong correlations or non-linear dependencies. The ISG method is also easy to implement, computationally cheap and would be relatively simple to include in automatically tuned codes as an alternative to the benchmark practice.

Cited by

• Preconditioning Hamiltonian Monte Carlo by minimizing Fisher Divergence
• High-dimensional Adaptive MCMC with Reduced Computational Complexity
• The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler
• Adaptive Riemannian Manifold Hamiltonian Monte Carlo with Hierarchical Metric

Related

• Entropy-based adaptive Hamiltonian Monte Carlo
• Faster Hamiltonian Monte Carlo by Learning Leapfrog Scale
• High-dimensional Adaptive MCMC with Reduced Computational Complexity
• Focusing on Difficult Directions for Learning HMC Trajectory Lengths
• A Condition Number for Hamiltonian Monte Carlo