2025/02/25 by Joris Dannemann, Dannemann, Joris, Gero Junike +1
Computer Science · Decision Sciences · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Fault Detection and Control Systems #Machine Learning (cs.LG) #Optimal Experimental Design Methods #Probability (math.PR) #VLSI and Analog Circuit Testing
paper · pdf · doi:10.48550/arxiv.2502.17913
openalex publication_date 2025/02/25 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28
Batch normalization is one of the most important regularization techniques for neural networks, significantly improving training by centering the layers of the neural network. There have been several attempts to provide a theoretical justification for batch ormalization. Santurkar and Tsipras (2018) [How does batch normalization help optimization? Advances in neural information rocessing systems, 31] claim that batch normalization improves initialization. We provide a counterexample showing that this claim s not true, i.e., batch normalization does not improve initialization.