2024/04/20 by Stefania Bellavia, Bellavia, Stefania, Benedetta Morini +3
Computer Science · Engineering · Mathematics · #Stochastic Gradient Optimization Techniques #Scheduling and Optimization Algorithms #Advanced Optimization Algorithms Research
paper · pdf · doi:10.48550/arxiv.2404.13382
This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [Curtis2019, Curtis2022]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [Bollapragada2018]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters.