2024/06/03 by Nicholas Pritchard, Nicholas J. Pritchard, A. Wicenec +1 · 1 voice · 4 citations
Computer Science · Decision Sciences · #Computer science #Database #Distributed and Parallel Computing Systems #Epistemology #Programming language #Reproducibility #Research Data Management Practices #Scientific Computing and Data Management #Software engineering #Theoretical computer science #Workflow #cs.DC
paper · pdf · open access · doi:10.1016/j.future.2024.107684
published in Future Generation Computer Systems 166, 107684 (Elsevier BV)
arxiv published 2024/06/03 · openalex publication_date 2024/12/20 · arxiv updated 2024/12/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Computational workflow management systems power contemporary data-intensive sciences. The slowly resolving reproducibility crisis presents both a sobering warning and an opportunity to iterate on what science and data processing entails. The Square Kilometre Array (SKA), the world’s largest radio telescope, is among the most extensive scientific projects underway and presents grand scientific collaboration and data-processing challenges. In this work, we aim to improve the ability of workflow management systems to facilitate reproducible, high-quality science. This work presents a scale and system-agnostic computational workflow model and extends five well-known reproducibility concepts into seven well-defined tenets for this workflow model. Additionally, we present a method to construct workflow execution signatures using cryptographic primitives in amortized constant time. We combine these three concepts and provide a concrete implementation in Data Activated Flow Graph Engine (DALiuGE), a workflow management system for the SKA to embed specific provenance information into workflow signatures, demonstrating the possibility of facilitating automatic formal verification of scientific quality in amortized constant time. We validate our approach with a simple yet representative astronomical processing task: filtering a noisy signal with a lowpass filter using CPU and GPU methods. This example shows the practicality and efficacy of combining formal tenet definitions with a workflow signature generation mechanism. Our framework, spanning formal UML specification, principled provenance information collection based on reproducibility tenets, and finally, a concrete example implementation in DALiuGE illuminates otherwise obscure scientific discrepancies and similarities between principally identical workflow executions. • Design of a scale and system-agnostic workflow model for reproducibility tests. • Defines seven reproducibility tests integrating goals found in the literature. • Describes and demonstrates a hash-graph-based workflow signature method. • Combined model, method and tests reveal numerical differences between workflows.