2021/10/26 by Andy Huynh, Huynh, Andy, Harshal A. Chaudhari +5 · 6 citations
Computer Science · #Cloud Computing and Resource Management #Advanced Data Storage Technologies #Advanced Database Systems and Queries
paper · pdf · doi:10.48550/arxiv.2110.13801
Log-Structured Merge trees (LSM trees) are increasingly used as the storage\nengines behind several data systems, frequently deployed in the cloud. Similar\nto other database architectures, LSM trees take into account information about\nthe expected workload (e.g., reads vs. writes, point vs. range queries) to\noptimize their performance via tuning. Operating in shared infrastructure like\nthe cloud, however, comes with a degree of workload uncertainty due to\nmulti-tenancy and the fast-evolving nature of modern applications. Systems with\nstatic tuning discount the variability of such hybrid workloads and hence\nprovide an inconsistent and overall suboptimal performance.\n To address this problem, we introduce Endure - a new paradigm for tuning LSM\ntrees in the presence of workload uncertainty. Specifically, we focus on the\nimpact of the choice of compaction policies, size-ratio, and memory allocation\non the overall performance. Endure considers a robust formulation of the\nthroughput maximization problem, and recommends a tuning that maximizes the\nworst-case throughput over a neighborhood of each expected workload.\nAdditionally, an uncertainty tuning parameter controls the size of this\nneighborhood, thereby allowing the output tunings to be conservative or\noptimistic. Through both model-based and extensive experimental evaluation of\nEndure in the state-of-the-art LSM-based storage engine, RocksDB, we show that\nthe robust tuning methodology consistently outperforms classical tun-ing\nstrategies. We benchmark Endure using 15 workload templates that generate more\nthan 10000 unique noisy workloads. The robust tunings output by Endure lead up\nto a 5\× improvement in through-put in presence of uncertainty. On the\nflip side, when the observed workload exactly matches the expected one, Endure\ntunings have negligible performance loss.\n