Stochastic gradient descent on Riemannian manifolds

2011/11/30 by Silvere Bonnabel · 4 citations
Computer Science · Mathematics · #cs.LG #math.OC #stat.ML

paper · pdf · doi:10.1109/tac.2013.2254619

published as IEEE Transactions on Automatic Control, Vol 58 (9), pages 2217 - 2229, Sept 2013 · A slightly shorter version has been published in IEEE Transactions Automatic Control

arxiv created 2013/11/19 · arxiv updated 2016/11/17

Abstract

Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four examples. In particular a novel gossip algorithm on the set of covariance matrices is derived and tested numerically.

Cited by

• Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks
• Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
• Revisit First-order Methods for Geodesically Convex Optimization
• An Adaptive Algorithm for Bilevel Optimization on Riemannian Manifolds
• Lattice: Learning to Efficiently Compress the Memory