Robust Generative Restricted Kernel Machines using Weighted Conjugate Feature Duality

2020/02/04 by Arun Pandey, Pandey, Arun, Joachim Schreurs +3
Computer Science · Mathematics · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2002.01180

arxiv created 2020/06/23 · arxiv updated 2020/06/24

Abstract

Interest in generative models has grown tremendously in the past decade. However, their training performance can be adversely affected by contamination, where outliers are encoded in the representation of the model. This results in the generation of noisy data. In this paper, we introduce weighted conjugate feature duality in the framework of Restricted Kernel Machines (RKMs). The RKM formulation allows for an easy integration of methods from classical robust statistics. This formulation is used to fine-tune the latent space of generative RKMs using a weighting function based on the Minimum Covariance Determinant, which is a highly robust estimator of multivariate location and scatter. Experiments show that the weighted RKM is capable of generating clean images when contamination is present in the training data. We further show that the robust method also preserves uncorrelated feature learning through qualitative and quantitative experiments on standard datasets.

Related

• CI-RKM: A Class-Informed Approach to Robust Restricted Kernel Machines
• Generative Restricted Kernel Machines: A Framework for Multi-view Generation and Disentangled Feature Learning
• Robust Kernel (Cross-) Covariance Operators in Reproducing Kernel Hilbert Space toward Kernel Methods
• Learning with invariances in random features and kernel models
• Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Density