2019/01/01 by Philipp Becker, Becker, Philipp, Harit Pandya +10 · 11 citations
Computer Science · Neuroscience · #Neural Networks and Applications #Gaussian Processes and Bayesian Inference #Functional Brain Connectivity Studies
paper · pdf · doi:10.48550/arxiv.1905.07357
In order to integrate uncertainty estimates into deep time-series modelling,\nKalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep\nlearning models, however, such approaches typically rely on approximate\ninference techniques such as variational inference which makes learning more\ncomplex and often less scalable due to approximation errors. We propose a new\ndeep approach to Kalman filtering which can be learned directly in an\nend-to-end manner using backpropagation without additional approximations. Our\napproach uses a high-dimensional factorized latent state representation for\nwhich the Kalman updates simplify to scalar operations and thus avoids hard to\nbackpropagate, computationally heavy and potentially unstable matrix\ninversions. Moreover, we use locally linear dynamic models to efficiently\npropagate the latent state to the next time step. The resulting network\narchitecture, which we call Recurrent Kalman Network (RKN), can be used for any\ntime-series data, similar to a LSTM (Hochreiter & Schmidhuber, 1997) but uses\nan explicit representation of uncertainty. As shown by our experiments, the RKN\nobtains much more accurate uncertainty estimates than an LSTM or Gated\nRecurrent Units (GRUs) (Cho et al., 2014) while also showing a slightly\nimproved prediction performance and outperforms various recent generative\nmodels on an image imputation task.\n