2020/04/29 by Evgeny M. Mirkes, Jeza Allohibi, Alexander N. Gorban +1 · 44 citations
Computer Science · Mathematics · #Blessing #Contrast (vision) #Curse #Curse of dimensionality #Euclidean distance #Euclidean geometry #Face and Expression Recognition #Imbalanced Data Classification Techniques #Machine Learning and Data Classification #Norm (philosophy) #cs.LG #stat.ML
paper · pdf · doi:10.3390/e22101105
published in Entropy 22(10), 1105 (Multidisciplinary Digital Publishing Institute)
arxiv created 2020/04/29 · openalex created_date 2020/05/13 · openalex publication_date 2020/09/30 · arxiv updated 2021/03/04 · openalex updated_date 2026/08/05
The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional lp quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm l2, but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for lp at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data.