2019/12/09 by Aljaž Božič, Božič, Aljaž, Michael Zollhöfer +5 · 3 citations
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR) #Optical measurement and interference techniques
paper · pdf · doi:10.48550/arxiv.1912.04302
openalex publication_date 2019/12/09 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
Applying data-driven approaches to non-rigid 3D reconstruction has been\ndifficult, which we believe can be attributed to the lack of a large-scale\ntraining corpus. Unfortunately, this method fails for important cases such as\nhighly non-rigid deformations. We first address this problem of lack of data by\nintroducing a novel semi-supervised strategy to obtain dense inter-frame\ncorrespondences from a sparse set of annotations. This way, we obtain a large\ndataset of 400 scenes, over 390,000 RGB-D frames, and 5,533 densely aligned\nframe pairs; in addition, we provide a test set along with several metrics for\nevaluation. Based on this corpus, we introduce a data-driven non-rigid feature\nmatching approach, which we integrate into an optimization-based reconstruction\npipeline. Here, we propose a new neural network that operates on RGB-D frames,\nwhile maintaining robustness under large non-rigid deformations and producing\naccurate predictions. Our approach significantly outperforms existing non-rigid\nreconstruction methods that do not use learned data terms, as well as\nlearning-based approaches that only use self-supervision.\n