Towards Adaptive Self-Normalized Importance Samplers

2025/05/01 by Nicola Branchini, Víctor Elvira, Branchini, Nicola +1 · 1 voice
#stat.CO

paper · pdf · doi:10.48550/arxiv.2505.00372

Abstract

The self-normalized importance sampling (SNIS) estimator is a Monte Carlo estimator widely used to approximate expectations in statistical signal processing and machine learning. The efficiency of SNIS depends on the choice of proposal, but selecting a good proposal is typically unfeasible. In particular, most of the existing adaptive IS (AIS) literature overlooks the optimal SNIS proposal. In this paper, we introduce an AIS framework that uses MCMC to approximate the optimal SNIS proposal within an iterative scheme. This is, to the best of our knowledge, the first AIS framework targeting specifically the SNIS optimal proposal. We find a close connection with adaptive schemes used in ratio importance sampling (RIS), which also brings a new perspective and paves the way for combining techniques from AIS and adaptive RIS. We outline possible extensions, connections with existing MCMC-driven AIS algorithms, theoretical directions, and demonstrate performance in numerical examples.

Citations

• The pseudo-marginal approach for efficient Monte Carlo computations

Discussions

• 🚨 New paper: “Towards Adaptive Self-Normalized IS”, @ IEEE Statistical Signal Processing Workshop. TLDR; To estimate µ = E_p[f(θ)] with SNIS, instead of doing MCMC on p(θ) or learning a parametric q( [bsky, 31 points, 1 comments]

Related

• BR-SNIS: Bias Reduced Self-Normalized Importance Sampling
• Generalizing self-normalized importance sampling with couplings
• Adaptively Optimised Adaptive Importance Samplers
• Gradient-based Adaptive Importance Samplers
• Stochastic adaptation of importance sampler