2026/04/26 by Sharif Ahmed, Nasir U. Eisty
Computer Science · Decision Sciences · #Scientific Computing and Data Management #Software Engineering Research #Software Testing and Debugging Techniques #cs.SE
paper · pdf · doi:10.1145/3815572.3815763
Accepted at The Platform for Advanced Scientific Computing (PASC) Conference 2026, Bern, Switzerland
arxiv created 2026/04/26 · openalex publication_date 2026/06/26 · openalex created_date 2026/06/27 · openalex updated_date 2026/07/29 · arxiv updated 2026/07/30
Context: Innovation thrives on scientific software, with useful code review feedback enhancing its correctness and impact. However, unlike general-purpose commercial and open-source software, the usefulness of code review feedback (CR comment) in scientific software remains largely unstudied. Objective: This paper aims to characterize the usefulness of CR comment in scientific opens ource software (Sci-OSS), leveraging existing research on useful CR comment. Method: To achieve this objective, we mine successful Sci-OSS from GitHub, analyze their CR comments with usefulness related features, and compare the findings from prior research on general-purpose commercial and open-source CR comments. Results: The investigation on the usefulness of CR comments in SciOSS confirms many characteristics that prior research identified in general-purpose software. For example, subjective or negative CR comments remain not useful for the Sci-OSS. We also find CR comments which receive negative emoji reactions have a very small correlation with not useful comments, whereas the positive emojis show mixed correlations. Importantly, 6-33% CR comments in Sci-OSS are not useful in our mined repositories. Conclusions: Our investigation into Sci-OSS extends findings from CR comments' usefulness research on general-purpose software, benefiting developers, scientists, and researchers in the Sci-OSS community.