2019/07/26 by Audrey Repetti, Repetti, Audrey, Yves Wiaux +1 · 2 citations
Computer Science · Engineering · #49M27 #65K10 #68U10 #68W25 #90C26 #90C59 #94A08 #FOS: Mathematics #Optimization and Control (math.OC) #Optimization and Variational Analysis #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1907.11486
openalex publication_date 2019/07/26 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
We present a forward-backward-based algorithm to minimize a sum of a\ndifferentiable function and a nonsmooth function, both being possibly\nnonconvex. The main contribution of this work is to consider the challenging\ncase where the nonsmooth function corresponds to a sum of non-convex functions,\nresulting from composition between a strictly increasing, concave,\ndifferentiable function and a convex nonsmooth function. The proposed variable\nmetric Composite Function Forward-Backward algorithm (C2FB) circumvents the\nexplicit, and often challenging, computation of the proximity operator of the\ncomposite functions through a majorize-minimize approach. Precisely, each\ncomposite function is majorized using a linear approximation of the\ndifferentiable function, which allows one to apply the proximity step only to\nthe sum of the nonsmooth functions. We prove the convergence of the algorithm\niterates to a critical point of the objective function leveraging the\nKurdyka- L ojasiewicz inequality. The convergence is guaranteed even if the\nproximity operators are computed inexactly, considering relative errors. We\nshow that the proposed approach is a generalization of reweighting methods,\nwith convergence guarantees. In particular, applied to the log-sum function,\nour algorithm reduces to a generalized version of the celebrated reweighted\n\ℓ1 method. Finally, we show through simulations on an image processing\nproblem that the proposed C2FB algorithm necessitates less iterations to\nconverge and leads to better critical points compared with traditional\nreweighting methods and classic forward-backward algorithms.\n