2021/06/11 by Alberto De Santis, De Santis, Alberto, Giampaolo Liuzzi +5
Computer Science · #Neural Networks and Applications #Machine Learning and ELM #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2106.06477
Training a large multilayer neural network can present many difficulties due\nto the large number of useless stationary points. These points usually attract\nthe minimization algorithm used during the training phase, which therefore\nresults inefficient. Extending some results proposed in literature for shallow\nnetworks, we propose the mathematical characterization of a class of such\nstationary points that arise in deep neural networks training. Availing such a\ndescription, we are able to define an incremental training algorithm that\navoids getting stuck in the region of attraction of these undesirable\nstationary points.\n