2021/07/27 by David Bonet, Bonet, David, Antonio Ortega +5 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM #Advanced Neural Network Applications
paper · pdf · doi:10.48550/arxiv.2107.12972
State-of-the-art neural network architectures continue to scale in size and\ndeliver impressive generalization results, although this comes at the expense\nof limited interpretability. In particular, a key challenge is to determine\nwhen to stop training the model, as this has a significant impact on\ngeneralization. Convolutional neural networks (ConvNets) comprise\nhigh-dimensional feature spaces formed by the aggregation of multiple channels,\nwhere analyzing intermediate data representations and the model's evolution can\nbe challenging owing to the curse of dimensionality. We present channel-wise\nDeepNNK (CW-DeepNNK), a novel channel-wise generalization estimate based on\nnon-negative kernel regression (NNK) graphs with which we perform local\npolytope interpolation on low-dimensional channels. This method leads to\ninstance-based interpretability of both the learned data representations and\nthe relationship between channels. Motivated by our observations, we use\nCW-DeepNNK to propose a novel early stopping criterion that (i) does not\nrequire a validation set, (ii) is based on a task performance metric, and (iii)\nallows stopping to be reached at different points for each channel. Our\nexperiments demonstrate that our proposed method has advantages as compared to\nthe standard criterion based on validation set performance.\n