2020/10/01 by Christina Pavlopoulou, Pavlopoulou, Christina, Michael J. Carey +3 · 1 citation
Computer Science · #Cloud Computing and Resource Management #Advanced Database Systems and Queries #Data Management and Algorithms
paper · pdf · doi:10.48550/arxiv.2010.00728
Query Optimization remains an open problem for Big Data Management Systems.\nTraditional optimizers are cost-based and use statistical estimates of\nintermediate result cardinalities to assign costs and pick the best plan.\nHowever, such estimates tend to become less accurate because of filtering\nconditions caused either from undetected correlations between multiple\npredicates local to a single dataset, predicates with query parameters, or\npredicates involving user-defined functions (UDFs). Consequently, traditional\nquery optimizers tend to ignore or miscalculate those settings, thus leading to\nsuboptimal execution plans. Given the volume of today's data, a suboptimal plan\ncan quickly become very inefficient.\n In this work, we revisit the old idea of runtime dynamic optimization and\nadapt it to a shared-nothing distributed database system, AsterixDB. The\noptimization runs in stages (re-optimization points), starting by first\nexecuting all predicates local to a single dataset. The intermediate result\ncreated from each stage is used to re-optimize the remaining query. This\nre-optimization approach avoids inaccurate intermediate result cardinality\nestimations, thus leading to much better execution plans. While it introduces\nthe overhead for materializing these intermediate results, our experiments show\nthat this overhead is relatively small and it is an acceptable price to pay\ngiven the optimization benefits. In fact, our experimental evaluation shows\nthat runtime dynamic optimization leads to much better execution plans as\ncompared to the current default AsterixDB plans as well as to plans produced by\nstatic cost-based optimization (i.e. based on the initial dataset statistics)\nand other state-of-the-art approaches.\n