2023/06/20 by Mathilde Leuridan, Leuridan, Mathilde, James Hawkes +7 · 1 citation
Computer Science · #68P20 #Computational Geometry (cs.CG) #Distributed and Parallel Computing Systems #E.1 #F.2.2 #FOS: Computer and information sciences #G.4 #H.3.1 #H.3.3 #Information Retrieval (cs.IR) #Interconnection Networks and Systems #J.2 #J.3 #Parallel Computing and Optimization Techniques
paper · doi:10.48550/arxiv.2306.11553
openalex publication_date 2023/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Data extraction algorithms on data hypercubes, or datacubes, are traditionally only capable of cutting boxes of data along the datacube axes. For many use cases however, this is not a sufficient approach and returns more data than users might actually need. This not only forces users to apply post-processing after extraction, but more importantly this consumes more I/O resources than is necessary. When considering very large datacubes from which users only want to extract small non-rectangular subsets, the box approach does not scale well. Indeed, with this traditional approach, I/O systems quickly reach capacity, trying to read and return unwanted data to users. In this paper, we propose a novel technique, based on computational geometry concepts, which instead carefully pre-selects the precise bytes of data which the user needs in order to then only read those from the datacube. As we discuss later on, this novel extraction method will considerably help scale access to large petabyte size data hypercubes in a variety of scientific fields.