Probabilistic Interpretation of Linear Solvers

2014/02/10 by Hennig, Philipp · 4 citations
#65F10 #90C53 #F.2.1 #FOS: Computer and information sciences #FOS: Mathematics #G.1.2 #G.1.3 #G.1.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Probability (math.PR)

paper · doi:10.48550/arxiv.1402.2058

Abstract

This manuscript proposes a probabilistic framework for algorithms that iteratively solve unconstrained linear problems Bx = b with positive definite B for x. The goal is to replace the point estimates returned by existing methods with a Gaussian posterior belief over the elements of the inverse of B, which can be used to estimate errors. Recent probabilistic interpretations of the secant family of quasi-Newton optimization algorithms are extended. Combined with properties of the conjugate gradient algorithm, this leads to uncertainty-calibrated methods with very limited cost overhead over conjugate gradients, a self-contained novel interpretation of the quasi-Newton and conjugate gradient algorithms, and a foundation for new nonlinear optimization methods.

Cited by

• The Multivariate Taylor Measure Function Space and a Generalization of Taylor's Theorem
• Scalable h-adaptive probabilistic solver for time-independent and time-dependent systems
• Stochastic Taylor expansion via Poisson point processes
• The Taylor Measure and its Applications

Related

• Probabilistic Linear Solvers: A Unifying View
• Probabilistic Linear Solvers for Machine Learning
• Probabilistic Iterative Methods for Linear Systems
• Statistical Properties of the Probabilistic Numeric Linear Solver BayesCG
• On the construction of probabilistic Newton-type algorithms