Convergence of the Exponentiated Gradient Method with Armijo Line Search

2017/12/22 by Yen-Huan Li, Volkan Cevher, Li, Yen-Huan +1 · 1 citation
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1712.08480

openalex publication_date 2017/12/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Consider the problem of minimizing a convex differentiable function on the probability simplex, spectrahedron, or set of quantum density matrices. We prove that the exponentiated gradient method with Armjo line search always converges to the optimum, if the sequence of the iterates possesses a strictly positive limit point (element-wise for the vector case, and with respect to the Lowner partial ordering for the matrix case). To the best our knowledge, this is the first convergence result for a mirror descent-type method that only requires differentiability. The proof exploits self-concordant likeness of the log-partition function, which is of independent interest.

Cited by

• Secure Analog Beamforming Design for Wireless Communication Systems With Movable Antennas

Related

• A General Convergence Result for the Exponentiated Gradient Method
• A General Convergence Result for Mirror Descent with Armijo Line Search
• On the Efficient Implementation of the Matrix Exponentiated Gradient Algorithm for Low-Rank Matrix Optimization
• No quantum speedup over gradient descent for non-smooth convex optimization
• Analysis of the Gradient Method with an Armijo-Wolfe Line Search on a Class of Nonsmooth Convex Functions