Structures of Neural Network Effective Theories

2023/05/03 by Banta, Ian, Cai, Tianji, Craig, Nathaniel +1 · 4 citations
#Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #High Energy Physics - Phenomenology (hep-ph) #High Energy Physics - Theory (hep-th) #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2305.02334

Abstract

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.

Cited by

• Fermions and Supersymmetry in Neural Network Field Theories
• Finite-Width Neural Tangent Kernels from Feynman Diagrams
• Viability of perturbative expansion for quantum field theories on neurons
• Analysis of Fourier Neural Operators via Effective Field Theory

Related

• Neural Network Field Theories: Non-Gaussianity, Actions, and Locality
• The edge of chaos: quantum field theory and deep neural networks
• Field theory for biophysical neural networks
• Effective Field Neural Network
• Statistical Field Theory and Neural Structures Dynamics III: Effective Action for Connectivities, Interactions and Emerging Collective States