Determining Principal Component Cardinality through the Principle of Minimum Description Length

2018/12/31 by Tavory, Ami
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1901.00059

Abstract

PCA (Principal Component Analysis) and its variants areubiquitous techniques for matrix dimension reduction and reduced-dimensionlatent-factor extraction. One significant challenge in using PCA, is thechoice of the number of principal components. The information-theoreticMDL (Minimum Description Length) principle gives objective compression-based criteria for model selection, but it is difficult to analytically applyits modern definition - NML (Normalized Maximum Likelihood) - to theproblem of PCA. This work shows a general reduction of NML prob-lems to lower-dimension problems. Applying this reduction, it boundsthe NML of PCA, by terms of the NML of linear regression, which areknown.

Related

• MML Probabilistic Principal Component Analysis
• Supervised Linear Dimension-Reduction Methods: Review, Extensions, and Comparisons
• Principal Component Analysis When n < p: Challenges and Solutions
• Generalized Principal Component Analysis
• Bayesian dimensionality reduction with PCA using penalized semi-integrated likelihood