Training and Serving Machine Learning Models at Scale

2022/11/10 by Luciano Baresi, Baresi, Luciano, Giovanni Quattrocchi +1
Business, Management and Accounting · Computer Science · Decision Sciences · #Big Data and Business Intelligence #Data Quality and Management #Data Stream Mining Techniques #FOS: Computer and information sciences #Software Engineering (cs.SE)

paper · pdf · doi:10.48550/arxiv.2211.05516

openalex publication_date 2022/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years, Web services are becoming more and more intelligent (e.g., in understanding user preferences) thanks to the integration of components that rely on Machine Learning (ML). Before users can interact (inference phase) with an ML-based service (ML-Service), the underlying ML model must learn (training phase) from existing data, a process that requires long-lasting batch computations. The management of these two, diverse phases is complex and meeting time and quality requirements can hardly be done with manual approaches. This paper highlights some of the major issues in managing ML-services in both training and inference modes and presents some initial solutions that are able to meet set requirements with minimum user inputs. A preliminary evaluation demonstrates that our solutions allow these systems to become more efficient and predictable with respect to their response time and accuracy.

Related

• ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems
• StraightLine: An End-to-End Resource-Aware Scheduler for Machine Learning Application Requests
• Automating the Training and Deployment of Models in MLOps by Integrating Systems with Machine Learning
• Machine learning 2.0 : Engineering Data Driven AI Products
• Representation as a Service