Information Geometry of Exponentiated Gradient: Convergence beyond L-Smoothness

2025/04/07 by Yara Elshiaty, Elshiaty, Yara, Ferdinand Vanmaele +3
Computer Science · Mathematics · Physics and Astronomy · #FOS: Mathematics #Numerical methods in inverse problems #Optimization and Control (math.OC) #Statistical Mechanics and Entropy #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2504.05136

openalex publication_date 2025/04/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

We study the minimization of smooth, possibly nonconvex functions over the positive orthant, a key setting in Poisson inverse problems, using the exponentiated gradient (EG) method. Interpreting EG as Riemannian gradient descent (RGD) with the e-Exp map from information geometry as a retraction, we prove global convergence under weak assumptions -- without the need for L-smoothness -- and finite termination of Riemannian Armijo line search. Numerical experiments, including an accelerated variant, highlight EG's practical advantages, such as faster convergence compared to RGD based on interior-point geometry.

Citations

• Global Convergence of Hager-Zhang type Riemannian Conjugate Gradient Method
• Sufficient Descent Riemannian Conjugate Gradient Method
• Convergence of the Exponentiated Gradient Method with Armijo Line Search
• Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation
• The Information Geometry of Mirror Descent
• Exponentiated Gradient versus Gradient Descent for Linear Predictors

Related

• Mirror Descent and Novel Exponentiated Gradient Algorithms Using Trace-Form Entropies and Deformed Logarithms
• A diffusion-map-based algorithm for gradient computation on manifolds and applications
• Exponentially concave functions and a new information geometry
• Efficiently escaping saddle points on manifolds
• Accelerated Bregmann divergence optimization with SMART: an information geometry point of view