2021/06/19 by Pingyi Luo, Qiang Huang, Luo, Pingyi +3
Computer Science · #Advanced Clustering Algorithms Research #Advanced Image and Video Retrieval Techniques #Caching and Content Delivery #Databases (cs.DB) #Distributed #FOS: Computer and information sciences #Parallel #and Cluster Computing (cs.DC) #cs.DB #cs.DC
paper · pdf · doi:10.48550/arxiv.2106.10515
11 pages, 7 figures
arxiv created 2021/06/19 · openalex publication_date 2021/06/19 · arxiv updated 2021/06/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we introduce a novel Generic distributEd clustEring frameworK (GEEK) beyond k-means clustering to process massive amounts of data. To deal with different data types, GEEK first converts data in the original feature space into a unified format of buckets; then, we design a new Seeding method based on simILar bucKets (SILK) to determine initial seeds. Compared with state-of-the-art seeding methods such as k-means++ and its variants, SILK can automatically identify the number of initial seeds based on the closeness of shared data objects in similar buckets instead of pre-specifying k. Thus, its time complexity is independent of k. With these well-selected initial seeds, GEEK only needs a one-pass data assignment to get the final clusters. We implement GEEK on a distributed CPU-GPU platform for large-scale clustering. We evaluate the performance of GEEK over five large-scale real-life datasets and show that GEEK can deal with massive data of different types and is comparable to (or even better than) many state-of-the-art customized GPU-based methods, especially in large k values.