2018/08/17 by Amir Daneshmand, Ying Sun, Daneshmand, Amir +7
Computer Science · Engineering · Mathematics · #Distributed #Distributed Control Multi-Agent Systems #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Parallel #Sparse and Compressive Sensing Techniques #and Cluster Computing (cs.DC) #cs.DC #cs.LG #math.OC
paper · pdf · doi:10.48550/arxiv.1808.05933
openalex publication_date 2018/08/17 · arxiv created 2019/03/05 · arxiv updated 2019/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper studies Dictionary Learning problems wherein the learning task is distributed over a multi-agent network, modeled as a time-varying directed graph. This formulation is relevant, for instance, in Big Data scenarios where massive amounts of data are collected/stored in different locations (e.g., sensors, clouds) and aggregating and/or processing all data in a fusion center might be inefficient or unfeasible, due to resource limitations, communication overheads or privacy issues. We develop a unified decentralized algorithmic framework for this class of nonconvex problems, which is proved to converge to stationary solutions at a sublinear rate. The new method hinges on Successive Convex Approximation techniques, coupled with a decentralized tracking mechanism aiming at locally estimating the gradient of the smooth part of the sum-utility. To the best of our knowledge, this is the first provably convergent decentralized algorithm for Dictionary Learning and, more generally, bi-convex problems over (time-varying) (di)graphs.