2019/10/22 by Patrick Esser, Esser, Patrick, Johannes Haux +3
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1910.10223
openalex publication_date 2019/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep generative models come with the promise to learn an explainable\nrepresentation for visual objects that allows image sampling, synthesis, and\nselective modification. The main challenge is to learn to properly model the\nindependent latent characteristics of an object, especially its appearance and\npose. We present a novel approach that learns disentangled representations of\nthese characteristics and explains them individually. Training requires only\npairs of images depicting the same object appearance, but no pose annotations.\nWe propose an additional classifier that estimates the minimal amount of\nregularization required to enforce disentanglement. Thus both representations\ntogether can completely explain an image while being independent of each other.\nPrevious methods based on adversarial approaches fail to enforce this\nindependence, while methods based on variational approaches lead to\nuninformative representations. In experiments on diverse object categories, the\napproach successfully recombines pose and appearance to reconstruct and\nretarget novel synthesized images. We achieve significant improvements over\nstate-of-the-art methods which utilize the same level of supervision, and reach\nperformances comparable to those of pose-supervised approaches. However, we can\nhandle the vast body of articulated object classes for which no pose\nmodels/annotations are available.\n