Linear and Quadratic Discriminant Analysis: Tutorial

2019/06/01 by Benyamin Ghojogh, Mark Crowley, Ghojogh, Benyamin +1 · 2 citations
Computer Science · Mathematics · #Advanced Statistical Methods and Models #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1906.02590

openalex publication_date 2019/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) as two fundamental classification methods in statistical and probabilistic learning. We start with the optimization of decision boundary on which the posteriors are equal. Then, LDA and QDA are derived for binary and multiple classes. The estimation of parameters in LDA and QDA are also covered. Then, we explain how LDA and QDA are related to metric learning, kernel principal component analysis, Mahalanobis distance, logistic regression, Bayes optimal classifier, Gaussian naive Bayes, and likelihood ratio test. We also prove that LDA and Fisher discriminant analysis are equivalent. We finally clarify some of the theoretical concepts with simulations we provide.

Citations

• Fitting A Mixture Distribution to Data: Tutorial
• Eigenvalue and Generalized Eigenvalue Problems: Tutorial

Cited by

• Geometry and dynamics of representations in a precisely balanced memory network related to olfactory cortex
• Quantifying Classifier Utility under Local Differential Privacy

Related

• Quadratic Discriminant Analysis under Moderate Dimension
• Asymptotic Generalization Bound of Fisher's Linear Discriminant Analysis
• Fisher and Kernel Fisher Discriminant Analysis: Tutorial
• High Dimensional Quadratic Discriminant Analysis: Optimality and Phase Transitions
• Minimally Informed Linear Discriminant Analysis: training an LDA model with unlabelled data