2024/01/18 by Staufer-Steinnocher, Petra, Fischer, Manfred M.
paper · doi:10.57938/90ad6ca2-1d97-42d4-b9c6-858b86f80f2e
In this contributon we evaluate on-line and off-line techniques to train a single <br/>hidden layer neural network classifier with logistic hidden and softmax output transfer <br/>functions on a multispectral pixel-by-pixel classification problem. In contrast to <br/>current practice a multiple class cross-entropy error function has been chosen as the <br/>function to be minimized. The non-linear diffierential equations cannot be solved in <br/>closed form. To solve for a set of locally minimizing parameters we use the gradient <br/>descent technique for parameter updating based upon the backpropagation technique <br/>for evaluating the partial derivatives of the error function with respect to the <br/>parameter weights. Empirical evidence shows that on-line and epoch-based gradient <br/>descent backpropagation fail to converge within 100,000 iterations, due to the fixed <br/>step size. Batch gradient descent backpropagation training is superior in terms of <br/>learning speed and convergence behaviour. Stochastic epoch-based training tends to <br/>be slightly more effective than on-line and batch training in terms of generalization <br/>performance, especially when the number of training examples is larger. Moreover, it <br/>is less prone to fall into local minima than on-line and batch modes of operation. (authors' abstract)