2022/09/22 by Jamie F. Mair, Mair, Jamie F., D. C. Rose +3
Computer Science · #Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Neural Networks and Applications #Statistical Mechanics (cond-mat.stat-mech) #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.2209.11116
openalex publication_date 2022/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In machine learning, there is renewed interest in neural network ensembles (NNEs), whereby predictions are obtained as an aggregate from a diverse set of smaller models, rather than from a single larger model. Here, we show how to define and train a NNE using techniques from the study of rare trajectories in stochastic systems. We define an NNE in terms of the trajectory of the model parameters under a simple, and discrete in time, diffusive dynamics, and train the NNE by biasing these trajectories towards a small time-integrated loss, as controlled by appropriate counting fields which act as hyperparameters. We demonstrate the viability of this technique on a range of simple supervised learning tasks. We discuss potential advantages of our trajectory sampling approach compared with more conventional gradient based methods.