2006/01/02 by Jens Oehlschlägel, Oehlschlägel, Jens · 1 citation
Computer Science · #Advanced Clustering Algorithms Research #Artificial Intelligence (cs.AI) #Artificial intelligence #Cluster analysis #Computer science #Data Management and Algorithms #Data mining #Database #FOS: Computer and information sciences #G.3 #I.5.3 #Image Retrieval and Classification Techniques #Model selection #Scalability #Selection (genetic algorithm) #cs.AI
paper · pdf · doi:10.48550/arxiv.cs/0601001
published in arXiv (Cornell University) (Cornell University) · Article (10 figures). Changes in 2nd version: dropped supplements in favor of better integrated presentation, better literature coverage, put into proper English. Author's website available via http://www.truecluster.com
openalex publication_date 2006/01/02 · arxiv created 2007/05/28 · arxiv updated 2009/12/01 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/05
Data-based classification is fundamental to most branches of science. While recent years have brought enormous progress in various areas of statistical computing and clustering, some general challenges in clustering remain: model selection, robustness, and scalability to large datasets. We consider the important problem of deciding on the optimal number of clusters, given an arbitrary definition of space and clusteriness. We show how to construct a cluster information criterion that allows objective model selection. Differing from other approaches, our truecluster method does not require specific assumptions about underlying distributions, dissimilarity definitions or cluster models. Truecluster puts arbitrary clustering algorithms into a generic unified (sampling-based) statistical framework. It is scalable to big datasets and provides robust cluster assignments and case-wise diagnostics. Truecluster will make clustering more objective, allows for automation, and will save time and costs. Free R software is available.