2025/07/21 by Andrew Gracyk, Gracyk, Andrew
Computer Science · Neuroscience · #FOS: Computer and information sciences #Functional Brain Connectivity Studies #Machine Learning (cs.LG) #Neural dynamics and brain function #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2507.15431
openalex publication_date 2025/07/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02
We offer a theoretical mathematical background through Lagrangian optimization on the unit hyperspherical manifold and its tangential structure. Our methods can be categorized as inexact since our methods are projection-based and since we will perturb the functional optimization with epsilon-type quantities. We draw connections to the attention mechanism and the Transformer since it exists as a flow map in the tangent fiber for each token along the high-dimensional unit sphere. Our motivation for this work is primarily twofold: we study the attention mechanism under its flow map and its relations to traditional calculus of variations and Lagrangian optimization; and we study a range of calculus of variations on the unit hypersphere that appeal to a broader mathematical lens in approximating, variational contexts.