2015/04/27 by Vipul Arora, Arora, Vipul, Laxmidhar Behera +3
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Machine Learning and ELM #Model Reduction and Neural Networks #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)
paper · pdf · doi:10.48550/arxiv.1504.07278
openalex publication_date 2015/04/27 · openalex created_date 2022/09/05 · openalex updated_date 2026/07/28
A control theoretic approach is presented in this paper for both batch and\ninstantaneous updates of weights in feed-forward neural networks. The popular\nHamilton-Jacobi-Bellman (HJB) equation has been used to generate an optimal\nweight update law. The remarkable contribution in this paper is that closed\nform solutions for both optimal cost and weight update can be achieved for any\nfeed-forward network using HJB equation in a simple yet elegant manner. The\nproposed approach has been compared with some of the existing best performing\nlearning algorithms. It is found as expected that the proposed approach is\nfaster in convergence in terms of computational time. Some of the benchmark\ntest data such as 8-bit parity, breast cancer and credit approval, as well as\n2D Gabor function have been used to validate our claims. The paper also\ndiscusses issues related to global optimization. The limitations of popular\ndeterministic weight update laws are critiqued and the possibility of global\noptimization using HJB formulation is discussed. It is hoped that the proposed\nalgorithm will bring in a lot of interest in researchers working in developing\nfast learning algorithms and global optimization.\n