2020/04/02 by Suriya Gunasekar, Gunasekar, Suriya, Blake Woodworth +3 · 3 citations
Biochemistry, Genetics and Molecular Biology · #Caveolin-1 and cellular processes #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2004.01025
openalex publication_date 2020/04/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a primal only derivation of Mirror Descent as a "partial" discretization of gradient flow on a Riemannian manifold where the metric tensor is the Hessian of the Mirror Descent potential. We contrast this discretization to Natural Gradient Descent, which is obtained by a "full" forward Euler discretization. This view helps shed light on the relationship between the methods and allows generalizing Mirror Descent to general Riemannian geometries, even when the metric tensor is \em not a Hessian, and thus there is no "dual."