Unraveling the Black Box of Neural Networks: A Dynamic Extremum Mapper

2025/07/05 by Shengjian Chen, Chen, Shengjian
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Neural Networks and Applications #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2507.03885

openalex publication_date 2025/07/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02

Abstract

We point out that neural networks are not black boxes, and their generalization stems from the ability to dynamically map a dataset to the extrema of the model function. We further prove that the number of extrema in a neural network is positively correlated with the number of its parameters. We then propose a new algorithm that is significantly different from back-propagation algorithm, which mainly obtains the values of parameters by solving a system of linear equations. Some difficult situations, such as gradient vanishing and overfitting, can be simply explained and dealt with in this framework.

Citations

• A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
• GAN Inversion: A Survey
• Deep Reinforcement Learning for Autonomous Driving: A Survey
• Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
• A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
• How Does Batch Normalization Help Optimization?
• Opening the Black Box of Deep Neural Networks via Information
• "Why Should I Trust You?": Explaining the Predictions of Any Classifier
• Deep Learning and the Information Bottleneck Principle
• Multilayer feedforward networks are universal approximators

Related

• Extremal learning: extremizing the output of a neural network in regression problems
• A Dynamical Systems Perspective on the Analysis of Neural Networks
• Just How Flexible are Neural Networks in Practice?
• On the High Symmetry of Neural Network Functions
• Just One Layer Norm Guarantees Stable Extrapolation