Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

2020/06/17 by Atanas Mirchev, Mirchev, Atanas, Baris Kayalibay +5 · 1 citation
Computer Science · Earth and Planetary Sciences · Engineering · #3D Surveying and Cultural Heritage #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.2006.10178

openalex publication_date 2020/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.

Citations

• A Recurrent Latent Variable Model for Sequential Data
• Auto-Encoding Variational Bayes
• Neural SLAM: Learning to Explore with External Memory
• Recurrent World Models Facilitate Policy Evolution
• Learning Latent Dynamics for Planning from Pixels
• A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
• Approximate Bayesian inference in spatial environments
• Variational Tracking and Prediction with Generative Disentangled\n State-Space Models
• Dream to Control: Learning Behaviors by Latent Imagination
• Learning to Fly via Deep Model-Based Reinforcement Learning
• Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Cited by

• GPL-SLAM: A Laser SLAM Framework with Gaussian Process Based Extended Landmarks

Related

• Approximate Bayesian inference in spatial environments
• CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth
• Robust Monocular Localization of Drones by Adapting Domain Maps to Depth Prediction Inaccuracies
• Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence