Kernel Entropy Component Analysis

2009/05/11 by Robert Jenssen, R. Jenssen · 5 citations
Computer Science · #Blind Source Separation Techniques #Face and Expression Recognition #Neural Networks and Applications

paper · doi:10.1109/tpami.2009.100

openalex publication_date 2009/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

We introduce kernel entropy component analysis (kernel ECA) as a new method for data transformation and dimensionality reduction. Kernel ECA reveals structure relating to the Renyi entropy of the input space data set, estimated via a kernel matrix using Parzen windowing. This is achieved by projections onto a subset of entropy preserving kernel principal component analysis (kernel PCA) axes. This subset does not need, in general, to correspond to the top eigenvalues of the kernel matrix, in contrast to the dimensionality reduction using kernel PCA. We show that kernel ECA may produce strikingly different transformed data sets compared to kernel PCA, with a distinct angle-based structure. A new spectral clustering algorithm utilizing this structure is developed with positive results. Furthermore, kernel ECA is shown to be an useful alternative for pattern denoising.

Citations

• Analysis of a complex of statistical variables into principal components.
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• On Estimation of a Probability Density Function and Mode

Cited by

• Maximum Entropy Snapshot Sampling for Reduced Basis Generation
• Kernel methods and their derivatives: Concept and perspectives for the earth system sciences
• Deep divergence-based approach to clustering
• Affective computing using speech and eye gaze: a review and bimodal system proposal for continuous affect prediction
• The geometry of kernelized spectral clustering

Related

• An eigenanalysis of data centering in machine learning
• An Infinite Dimensional Analysis of Kernel Principal Components
• Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clustering
• Neural Network Characterization and Entropy Regulated Data Balancing through Principal Component Analysis
• Noncommutative Model Selection for Data Clustering and Dimension Reduction Using Relative von Neumann Entropy