2018/07/01 by Francesco Pace, Dimitrios Milios, Pace, Francesco +7
Computer Science · Engineering · #Advanced Data Storage Technologies #Business #Cloud Computing and Resource Management #Cluster (spacecraft) #Computer network #Computer science #Control (management) #Data Stream Mining Techniques #Distributed #Distributed computing #Engineering #FOS: Computer and information sciences #Finance #Operations research #Order (exchange) #Parallel #Reservation #Resource (disambiguation) #Resource allocation #Turnaround time #and Cluster Computing (cs.DC) #cs.DC
paper · pdf · doi:10.48550/arxiv.1807.00368
arxiv created 2018/07/01 · openalex publication_date 2018/07/01 · arxiv updated 2018/07/03 · openalex created_date 2018/07/10 · openalex updated_date 2026/07/28
Nowadays, data-centers are largely under-utilized because resource allocation is based on reservation mechanisms which ignore actual resource utilization. Indeed, it is common to reserve resources for peak demand, which may occur only for a small portion of the application life time. As a consequence, cluster resources often go under-utilized. In this work, we propose a mechanism that improves cluster utilization, thus decreasing the average turnaround time, while preventing application failures due to contention in accessing finite resources such as RAM. Our approach monitors resource utilization and employs a data-driven approach to resource demand forecasting, featuring quantification of uncertainty in the predictions. Using demand forecast and its confidence, our mechanism modulates cluster resources assigned to running applications, and reduces the turnaround time by more than one order of magnitude while keeping application failures under control. Thus, tenants enjoy a responsive system and providers benefit from an efficient cluster utilization.