2016/06/21 by Pooyan Jamshidi, Jamshidi, Pooyan, Giuliano Casale +1 · 4 citations
Computer Science · #Advanced Multi-Objective Optimization Algorithms #Data Stream Mining Techniques #Distributed #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Parallel #and Cluster Computing (cs.DC)
paper · doi:10.48550/arxiv.1606.06543
openalex publication_date 2016/06/21 · openalex created_date 2022/10/05 · openalex updated_date 2026/07/28
Finding optimal configurations for Stream Processing Systems (SPS) is a challenging problem due to the large number of parameters that can influence their performance and the lack of analytical models to anticipate the effect of a change. To tackle this issue, we consider tuning methods where an experimenter is given a limited budget of experiments and needs to carefully allocate this budget to find optimal configurations. We propose in this setting Bayesian Optimization for Configuration Optimization (BO4CO), an auto-tuning algorithm that leverages Gaussian Processes (GPs) to iteratively capture posterior distributions of the configuration spaces and sequentially drive the experimentation. Validation based on Apache Storm demonstrates that our approach locates optimal configurations within a limited experimental budget, with an improvement of SPS performance typically of at least an order of magnitude compared to existing configuration algorithms.