2019/09/19 by José A. Carrillo, Carrillo, José A., Shi Jin +5 · 5 citations
Computer Science · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #Distributed Control Multi-Agent Systems #FOS: Mathematics #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1909.09249
openalex publication_date 2019/09/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30
We improve recently introduced consensus-based optimization method, proposed in [R. Pinnau, C. Totzeck, O. Tse and S. Martin, Math. Models Methods Appl. Sci., 27(01):183--204, 2017], which is a gradient-free optimization method for general non-convex functions. We first replace the isotropic geometric Brownian motion by the component-wise one, thus removing the dimensionality dependence of the drift rate, making the method more competitive for high dimensional optimization problems. Secondly, we utilize the random mini-batch ideas to reduce the computational cost of calculating the weighted average which the individual particles tend to relax toward. For its mean-field limit--a nonlinear Fokker-Planck equation--we prove, in both time continuous and semi-discrete settings, that the convergence of the method, which is exponential in time, is guaranteed with parameter constraints \it independent of the dimensionality. We also conduct numerical tests to high dimensional problems to check the success rate of the method.