The Gradient descent method from the perspective of fractional calculus

2019/11/19 by Hai, Pham Viet, Rosenfeld, Joel A.
#FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.1911.08058

Abstract

Motivated by gradient methods in optimization theory, we give methods based on ψ-fractional derivatives of order α in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving ψ-fractional derivatives. Numerical examples using the ABM method show that the fractional order α and weight ψ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.

Related

• Convergence Analysis of Fractional Gradient Descent
• A novel perspective to gradient method: the fractional order approach
• On optimization and solution of roots of a function using Taylor's expansion and fractional derivatives
• A Conjugate Gradient Method for Nonlinear Programming Problems using Caputo Fractional Gradients
• Applications of fractional calculus in learned optimization