2019/08/07 by Brian R. Swenson, Swenson, Brian, Ryan Murray +5 · 1 citation
Computer Science · Mathematics · #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Multiagent Systems (cs.MA) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1908.02747
openalex publication_date 2019/08/07 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
The paper studies a distributed gradient descent (DGD) process and considers\nthe problem of showing that in nonconvex optimization problems, DGD typically\nconverges to local minima rather than saddle points. The paper considers\nunconstrained minimization of a smooth objective function. In centralized\nsettings, the problem of demonstrating nonconvergence to saddle points of\ngradient descent (and variants) is typically handled by way of the\nstable-manifold theorem from classical dynamical systems theory. However, the\nclassical stable-manifold theorem is not applicable in distributed settings.\nThe paper develops an appropriate stable-manifold theorem for DGD showing that\nconvergence to saddle points may only occur from a low-dimensional stable\nmanifold. Under appropriate assumptions (e.g., coercivity), this result implies\nthat DGD typically converges to local minima and not to saddle points.\n