Gradient descent in some simple settings

2018/08/14 by Y. Cooper, Cooper, Y. · 1 citation
Computer Science · Mathematics · #Advanced Mathematical Modeling in Engineering #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Mathematics #Geometric Analysis and Curvature Flows #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Nonlinear Partial Differential Equations #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.1808.04839

openalex publication_date 2018/08/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this note, we observe the behavior of gradient flow and discrete and noisy gradient descent in some simple settings. It is commonly noted that addition of noise to gradient descent can affect the trajectory of gradient descent. Here, we run some computer experiments for gradient descent on some simple functions, and observe this principle in some concrete examples.

Cited by

• Discretized Gradient Flow for Manifold Learning in the Space of Embeddings

Related

• Gradient descent in higher codimension
• From Optimization to Sampling Through Gradient Flows
• Gradient Descent Can Take Exponential Time to Escape Saddle Points
• Understanding the unstable convergence of gradient descent
• Gradient directions and relative inexactness in optimization and machine learning