2020/01/27 by Yisel Garí, David A. Monge, Garí, Yisel +7
Computer Science · #C.2.4 #Cloud Computing and Resource Management #Distributed #FOS: Computer and information sciences #I.2.11 #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2001.09957
openalex publication_date 2020/01/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Reinforcement Learning (RL) has demonstrated a great potential for\nautomatically solving decision-making problems in complex uncertain\nenvironments. RL proposes a computational approach that allows learning through\ninteraction in an environment with stochastic behavior, where agents take\nactions to maximize some cumulative short-term and long-term rewards. Some of\nthe most impressive results have been shown in Game Theory where agents\nexhibited superhuman performance in games like Go or Starcraft 2, which led to\nits gradual adoption in many other domains, including Cloud Computing.\nTherefore, RL appears as a promising approach for Autoscaling in Cloud since it\nis possible to learn transparent (with no human intervention), dynamic (no\nstatic plans), and adaptable (constantly updated) resource management policies\nto execute applications. These are three important distinctive aspects to\nconsider in comparison with other widely used autoscaling policies that are\ndefined in an ad-hoc way or statically computed as in solutions based on\nmeta-heuristics. Autoscaling exploits the Cloud elasticity to optimize the\nexecution of applications according to given optimization criteria, which\ndemands to decide when and how to scale-up/down computational resources, and\nhow to assign them to the upcoming processing workload. Such actions have to be\ntaken considering that the Cloud is a dynamic and uncertain environment.\nMotivated by this, many works apply RL to the autoscaling problem in the Cloud.\nIn this work, we survey exhaustively those proposals from major venues, and\nuniformly compare them based on a set of proposed taxonomies. We also discuss\nopen problems and prospective research in the area.\n