Diffusion Operator Geometry of Feedforward Representations

2026/05/31 by Kanishka Reddy · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #cond-mat.dis-nn #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2605.01107

arxiv created 2026/08/06 · arxiv updated 2026/08/07

Abstract

Feedforward neural networks transform data through learned representations whose geometry shapes how classes separate and relate across successive layers. We study that geometry through diffusion operators. Each feature-cloud snapshot is assigned a Gaussian-kernel Markov operator, giving a smooth description of one-step transport between classes from which spectral, boundary, and local geometric information can be read. We define the empirical class chain, state the condition under which it is an exact Markov quotient, and derive both the corresponding population transition and a simpler overlap chain based on expected class affinities. For balanced shared-covariance Gaussian class-conditional snapshots these affinities have closed forms controlled by a regularized Mahalanobis separation, which yields explicit expressions for leakage and coarse spectral behaviour. We further show that operator observables vary smoothly under feature perturbations, whereas hard neighborhood graphs are controlled by neighbor-order margins. Experiments on CIFAR-10 and CIFAR-100 ResNet-18 representations find that class transport becomes increasingly persistent with depth while retaining structured relations between classes, and that the diffusion class chain is more stable than its k-nearest-neighbor counterpart under matched perturbations.

Citations

• Computing Diffusion Geometry
• The Mean-Field Dynamics of Transformers
• Neural Feature Geometry Evolves as Discrete Ricci Flow
• Exploring the Manifold of Neural Networks Using Diffusion Geometry
• Clustering in Causal Attention Masking
• Manifold Diffusion Geometry: Curvature, Tangent Spaces, and Dimension
• Deep Learning as Ricci Flow
• A mathematical perspective on Transformers
• Continuum Limits of Ollivier's Ricci Curvature on data clouds: pointwise consistency and global lower bounds
• The emergence of clusters in self-attention dynamics
• Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks
• Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path
• A Geometric Analysis of Neural Collapse with Unconstrained Features
• Prevalence of Neural Collapse during the terminal phase of deep learning training
• Ollivier-Ricci curvature convergence in random geometric graphs
• Lipschitz regularity of graph Laplacians on random data clouds
• Topology of deep neural networks
• Community Detection on Networks with Ricci Flow
• Intrinsic dimension of data representations in deep neural networks
• Similarity of Neural Network Representations Revisited
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Insights on representational similarity in neural networks with canonical correlation
• Deep Neural Networks as Gaussian Processes
• Data-driven model reduction and transfer operator approximation
• Deep Information Propagation
• Understanding intermediate layers using linear classifier probes
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
• Deep Residual Learning for Image Recognition
• Ricci curvature of graphs
• An Analysis of the Convergence of Graph Laplacians
• A Tutorial on Spectral Clustering
• A tutorial on spectral clustering
• Ricci curvature of Markov chains on metric spaces
• Ricci curvature of Markov chains on metric spaces
• Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• Bochner's Method for Cell Complexes and Combinatorial Ricci Curvature
• Diffusion maps