2024/11/06 by Tharindu Adikari, Adikari, Tharindu, Haider Al-Lawati +7
Computer Science · #Cloud Computing and Resource Management #Data Stream Mining Techniques #Distributed #FOS: Computer and information sciences #IoT and Edge/Fog Computing #Parallel #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2411.03645
openalex publication_date 2024/11/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider the problem of stragglers in distributed computing systems. Stragglers, which are compute nodes that unpredictably slow down, often increase the completion times of tasks. One common approach to mitigating stragglers is work replication, where only the first completion among replicated tasks is accepted, discarding the others. However, discarding work leads to resource wastage. In this paper, we propose a method for exploiting the work completed by stragglers rather than discarding it. The idea is to increase the granularity of the assigned work, and to increase the frequency of worker updates. We show that the proposed method reduces the completion time of tasks via experiments performed on a simulated cluster as well as on Amazon EC2 with Apache Hadoop.