SGPRS: Seamless GPU Partitioning Real-Time Scheduler for Periodic Deep Learning Workloads

2024/04/13 by Amir Fakhim Babaei, Babaei, Amir Fakhim, Thidapat Chantem +1
Computer Science · #Digital Image Processing Techniques #Distributed #Embedded Systems Design Techniques #FOS: Computer and information sciences #FOS: Electrical engineering #Parallel #Parallel Computing and Optimization Techniques #Software Engineering (cs.SE) #Systems and Control (eess.SY) #and Cluster Computing (cs.DC) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2406.09425

openalex publication_date 2024/04/13 · openalex created_date 2024/06/18 · openalex updated_date 2026/07/28

Abstract

Deep Neural Networks (DNNs) are useful in many applications, including transportation, healthcare, and speech recognition. Despite various efforts to improve accuracy, few works have studied DNN in the context of real-time requirements. Coarse resource allocation and sequential execution in existing frameworks result in underutilization. In this work, we conduct GPU speedup gain analysis and propose SGPRS, the first real-time GPU scheduler considering zero configuration partition switch. The proposed scheduler not only meets more deadlines for parallel tasks but also sustains overall performance beyond the pivot point.

Related

• DARIS: An Oversubscribed Spatio-Temporal Scheduler for Real-Time DNN Inference on GPUs
• D-STACK: High Throughput DNN Inference by Effective Multiplexing and Spatio-Temporal Scheduling of GPUs
• Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision
• Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem
• Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning