Kernel Alignment Inspired Linear Discriminant Analysis

2016/10/14 by Shuai Zheng, Zheng, Shuai, Chris Ding +1
Computer Science · Engineering · #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Remote-Sensing Image Classification #Text and Document Classification Technologies

paper · pdf · doi:10.48550/arxiv.1610.04576

openalex publication_date 2016/10/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Kernel alignment measures the degree of similarity between two kernels. In this paper, inspired from kernel alignment, we propose a new Linear Discriminant Analysis (LDA) formulation, kernel alignment LDA (kaLDA). We first define two kernels, data kernel and class indicator kernel. The problem is to find a subspace to maximize the alignment between subspace-transformed data kernel and class indicator kernel. Surprisingly, the kernel alignment induced kaLDA objective function is very similar to classical LDA and can be expressed using between-class and total scatter matrices. This can be extended to multi-label data. We use a Stiefel-manifold gradient descent algorithm to solve this problem. We perform experiments on 8 single-label and 6 multi-label data sets. Results show that kaLDA has very good performance on many single-label and multi-label problems.

Citations

• Gradient-based learning applied to document recognition

Related

• Kernel Alignment for Unsupervised Feature Selection via Matrix Factorization
• Accelerated kernel discriminant analysis
• Alignment Based Kernel Learning with a Continuous Set of Base Kernels
• Revisiting Classical Multiclass Linear Discriminant Analysis with a Novel Prototype-based Interpretable Solution
• Manifold Approximation leads to Robust Kernel Alignment