2024/01/18 by Fischer, Manfred M., Staufer-Steinnocher, Petra
paper · doi:10.57938/5b7541c1-fefe-4eb1-af1a-34b4aae86d28
Various techniques of optimizing the multiple class cross-entropy error function <br/>to train single hidden layer neural network classifiers with softmax output transfer <br/>functions are investigated on a real-world multispectral pixel-by-pixel classification <br/>problem that is of fundamental importance in remote sensing. These techniques <br/>include epoch-based and batch versions of backpropagation of gradient descent, <br/>PR-conjugate gradient and BFGS quasi-Newton errors. The method of choice <br/>depends upon the nature of the learning task and whether one wants to optimize <br/>learning for speed or generalization performance. It was found that, comparatively <br/>considered, gradient descent error backpropagation provided the best and most stable <br/>out-of-sample performance results across batch and epoch-based modes of operation. <br/>If the goal is to maximize learning speed and a sacrifice in generalisation is acceptable, <br/>then PR-conjugate gradient error backpropagation tends to be superior. If the <br/>training set is very large, stochastic epoch-based versions of local optimizers should <br/>be chosen utilizing a larger rather than a smaller epoch size to avoid inacceptable <br/>instabilities in the generalization results. (authors' abstract)