Regularizers versus Losses for Nonlinear Dimensionality Reduction: A Factored View with New Convex Relaxations

2012/06/27 by Yaoliang Yu, Yu, Yaoliang, James Neufeld +7
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical methods in inverse problems #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.1206.6455

openalex publication_date 2012/06/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We demonstrate that almost all non-parametric dimensionality reduction methods can be expressed by a simple procedure: regularized loss minimization plus singular value truncation. By distinguishing the role of the loss and regularizer in such a process, we recover a factored perspective that reveals some gaps in the current literature. Beyond identifying a useful new loss for manifold unfolding, a key contribution is to derive new convex regularizers that combine distance maximization with rank reduction. These regularizers can be applied to any loss.

Citations

• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
• Robust principal component analysis?

Related

• On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding
• Bridging linearity-based and kernel-based sufficient dimension reduction
• Kernel manifolds: nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces
• A Reduced-Space Algorithm for Minimizing ℓ₁-Regularized Convex Functions
• Geometric Regularization from Overparameterization