A Bayesian Characterization of Ensemble Kalman Updates

2025/09/30 by Jorgensen, Frederic J. N., Marzouk, Youssef M. · 1 citation
#62F15 #65C35 #93E11 #FOS: Mathematics #G.3 #I.6.5 #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Probability (math.PR) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2510.00158

Abstract

The update in the Ensemble Kalman Filter (EnKF), called the Ensemble Kalman Update (EnKU), is widely used for Bayesian inference in inverse problems and data assimilation. At each filtering step, it approximates the solution to a likelihood-free Bayesian inversion from an ensemble of particles (X_i,Y_i)∼π sampled from a joint measure π and an observation y_∗∈ℝ^m. The posterior π_{X|Y=y_∗} is approximated by transporting (X_i,Y_i) through an affine map L^EnKU_{y_∗}(x,y) determined by the Kalman gain. While the EnKU is exact for Gaussian joints π in the mean-field limit, exactness alone does not fix the update: infinitely many affine maps L_{y_∗} push a Gaussian π to π_{X|Y=y_∗}. This raises a question: which affine map should estimate the posterior? We provide a characterization of the EnKU among all such maps. First, we describe the set E^EnKU of laws where the EnKU yields exact conditioning, showing it is larger than the Gaussian family. Next, we prove that, except for a small class of highly symmetric distributions in E^EnKU (including Gaussians), the EnKU is the unique exact affine conditioning map. Finally, we ask for the largest possible set F where any measure-dependent affine transport could be exact; after characterizing F, we show the EnKU's exactness set is almost maximal: F=E^EnKU\cupS_nl-dec, where S_nl-dec is a small symmetry class. Thus, among affine transports, the EnKU is near-optimal for exact conditioning beyond Gaussians and is the unique affine update achieving exactness for any measure in F except a subclass of strongly symmetric laws.

Citations

• Learning Enhanced Ensemble Filters
• An adaptive ensemble filter for heavy-tailed distributions: tuning-free inflation and localization
• Ensemble Kalman Methods: A Mean Field Perspective
• Learning to Assimilate in Chaotic Dynamical Systems
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
• Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation
• A note on the formulation of the Ensemble Adjustment Kalman Filter
• Coupling techniques for nonlinear ensemble filtering
• Convergence Analysis of Ensemble Kalman Inversion: The Linear, Noisy Case
• On the stability and the uniform propagation of chaos properties of\n Ensemble Kalman-Bucy filters
• Analysis of the ensemble Kalman filter for inverse problems
• Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter

Cited by

• Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes

Related

• Convergence Acceleration of Ensemble Kalman Inversion in Nonlinear Settings
• The Ensemble Kalman Update is an Empirical Matheron Update
• Analysis of the ensemble Kalman filter for marginal and joint posteriors
• A Brief Tutorial on the Ensemble Kalman Filter
• The Mean Field Ensemble Kalman Filter: Near-Gaussian Setting