2020/05/29 by Lucia Schuler, Schuler, Lucia, Somaya Jamil +3
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Cloud Computing and Resource Management #Distributed #FOS: Computer and information sciences #IoT and Edge/Fog Computing #Machine Learning (cs.LG) #Parallel #Retinal Imaging and Analysis #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2005.14410
openalex publication_date 2020/05/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Serverless computing has emerged as a compelling new paradigm of cloud\ncomputing models in recent years. It promises the user services at large scale\nand low cost while eliminating the need for infrastructure management. On cloud\nprovider side, flexible resource management is required to meet fluctuating\ndemand. It can be enabled through automated provisioning and deprovisioning of\nresources. A common approach among both commercial and open source serverless\ncomputing platforms is workload-based auto-scaling, where a designated\nalgorithm scales instances according to the number of incoming requests. In the\nrecently evolving serverless framework Knative a request-based policy is\nproposed, where the algorithm scales resources by a configured maximum number\nof requests that can be processed in parallel per instance, the so-called\nconcurrency. As we show in a baseline experiment, this predefined concurrency\nlevel can strongly influence the performance of a serverless application.\nHowever, identifying the concurrency configuration that yields the highest\npossible quality of service is a challenging task due to various factors, e.g.\nvarying workload and complex infrastructure characteristics, influencing\nthroughput and latency. While there has been considerable research into\nintelligent techniques for optimizing auto-scaling for virtual machine\nprovisioning, this topic has not yet been discussed in the area of serverless\ncomputing. For this reason, we investigate the applicability of a reinforcement\nlearning approach, which has been proven on dynamic virtual machine\nprovisioning, to request-based auto-scaling in a serverless framework. Our\nresults show that within a limited number of iterations our proposed model\nlearns an effective scaling policy per workload, improving the performance\ncompared to the default auto-scaling configuration.\n