2020/07/30 by Nicolai Häni, Häni, Nicolai, Selim Engin +5
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Video Surveillance and Tracking Methods
paper · pdf · doi:10.48550/arxiv.2007.15627
Novel View Synthesis (NVS) is concerned with synthesizing views under camera\nviewpoint transformations from one or multiple input images. NVS requires\nexplicit reasoning about 3D object structure and unseen parts of the scene to\nsynthesize convincing results. As a result, current approaches typically rely\non supervised training with either ground truth 3D models or multiple target\nimages. We propose Continuous Object Representation Networks (CORN), a\nconditional architecture that encodes an input image's geometry and appearance\nthat map to a 3D consistent scene representation. We can train CORN with only\ntwo source images per object by combining our model with a neural renderer. A\nkey feature of CORN is that it requires no ground truth 3D models or target\nview supervision. Regardless, CORN performs well on challenging tasks such as\nnovel view synthesis and single-view 3D reconstruction and achieves performance\ncomparable to state-of-the-art approaches that use direct supervision. For\nup-to-date information, data, and code, please see our project page:\nhttps://nicolaihaeni.github.io/corn/.\n