2020/12/02 by Aljaž Božič, Božič, Aljaž, Pablo Palafox +9 · 4 citations
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR) #Human Pose and Action Recognition #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2012.01451
openalex publication_date 2020/12/02 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
We introduce Neural Deformation Graphs for globally-consistent deformation\ntracking and 3D reconstruction of non-rigid objects. Specifically, we\nimplicitly model a deformation graph via a deep neural network. This neural\ndeformation graph does not rely on any object-specific structure and, thus, can\nbe applied to general non-rigid deformation tracking. Our method globally\noptimizes this neural graph on a given sequence of depth camera observations of\na non-rigidly moving object. Based on explicit viewpoint consistency as well as\ninter-frame graph and surface consistency constraints, the underlying network\nis trained in a self-supervised fashion. We additionally optimize for the\ngeometry of the object with an implicit deformable multi-MLP shape\nrepresentation. Our approach does not assume sequential input data, thus\nenabling robust tracking of fast motions or even temporally disconnected\nrecordings. Our experiments demonstrate that our Neural Deformation Graphs\noutperform state-of-the-art non-rigid reconstruction approaches both\nqualitatively and quantitatively, with 64% improved reconstruction and 62%\nimproved deformation tracking performance.\n