2024/01/10 by Maren Raus, Raus, Maren, Yara Elshiaty +3
Engineering · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Statistical Mechanics and Entropy #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.2401.05196
openalex publication_date 2024/01/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
We investigate the problem of minimizing Kullback-Leibler divergence between a linear model Ax and a positive vector b in different convex domains (positive orthant, n-dimensional box, probability simplex). Our focus is on the SMART method that employs efficient multiplicative updates. We explore the exponentiated gradient method, which can be viewed as a Bregman proximal gradient method and as a Riemannian gradient descent on the parameter manifold of a corresponding distribution of the exponential family. This dual interpretation enables us to establish connections and achieve accelerated SMART iterates while smoothly incorporating constraints. The performance of the proposed acceleration schemes is demonstrated by large-scale numerical examples.