2017/08/30 by Vikramjit Mitra, Mitra, Vikramjit, Horacio Franco +1
Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing #Non-Destructive Testing Techniques #Speech Recognition and Synthesis #Speech and Audio Processing #Ultrasonics and Acoustic Wave Propagation
paper · pdf · doi:10.48550/arxiv.1708.09516
openalex publication_date 2017/08/30 · openalex created_date 2022/09/05 · openalex updated_date 2026/07/28
Unseen data conditions can inflict serious performance degradation on systems\nrelying on supervised machine learning algorithms. Because data can often be\nunseen, and because traditional machine learning algorithms are trained in a\nsupervised manner, unsupervised adaptation techniques must be used to adapt the\nmodel to the unseen data conditions. However, unsupervised adaptation is often\nchallenging, as one must generate some hypothesis given a model and then use\nthat hypothesis to bootstrap the model to the unseen data conditions.\nUnfortunately, reliability of such hypotheses is often poor, given the mismatch\nbetween the training and testing datasets. In such cases, a model hypothesis\nconfidence measure enables performing data selection for the model adaptation.\nUnderlying this approach is the fact that for unseen data conditions, data\nvariability is introduced to the model, which the model propagates to its\noutput decision, impacting decision reliability. In a fully connected network,\nthis data variability is propagated as distortions from one layer to the next.\nThis work aims to estimate the propagation of such distortion in the form of\nnetwork activation entropy, which is measured over a short- time running window\non the activation from each neuron of a given hidden layer, and these\nmeasurements are then used to compute summary entropy. This work demonstrates\nthat such an entropy measure can help to select data for unsupervised model\nadaptation, resulting in performance gains in speech recognition tasks. Results\nfrom standard benchmark speech recognition tasks show that the proposed\napproach can alleviate the performance degradation experienced under unseen\ndata conditions by iteratively adapting the model to the unseen datas acoustic\ncondition.\n