2021/04/13 by Chen-Hsuan Lin, Lin, Chen-Hsuan, Wei-Chiu Ma +5 · 74 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR) #Machine Learning (cs.LG) #Robotics (cs.RO) #Robotics and Sensor-Based Localization #cs.CV #cs.GR #cs.LG #cs.RO
paper · pdf · doi:10.48550/arxiv.2104.06405
Accepted to ICCV 2021 as oral presentation (project page & code: https://chenhsuanlin.bitbucket.io/bundle-adjusting-NeRF)
openalex publication_date 2021/04/13 · arxiv created 2021/08/19 · arxiv updated 2021/08/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Neural Radiance Fields (NeRF) have recently gained a surge of interest within the computer vision community for its power to synthesize photorealistic novel views of real-world scenes. One limitation of NeRF, however, is its requirement of accurate camera poses to learn the scene representations. In this paper, we propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect (or even unknown) camera poses -- the joint problem of learning neural 3D representations and registering camera frames. We establish a theoretical connection to classical image alignment and show that coarse-to-fine registration is also applicable to NeRF. Furthermore, we show that naïvely applying positional encoding in NeRF has a negative impact on registration with a synthesis-based objective. Experiments on synthetic and real-world data show that BARF can effectively optimize the neural scene representations and resolve large camera pose misalignment at the same time. This enables view synthesis and localization of video sequences from unknown camera poses, opening up new avenues for visual localization systems (e.g. SLAM) and potential applications for dense 3D mapping and reconstruction.