2019/08/28 by Gregory Kiar, Pablo Castro, Kiar, Gregory +10 · 1 citation
Engineering · Neuroscience · #EEG and Brain-Computer Interfaces #FOS: Biological sciences #FOS: Electrical engineering #Ferroelectric and Negative Capacitance Devices #Functional Brain Connectivity Studies #Image and Video Processing (eess.IV) #Neurons and Cognition (q-bio.NC) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1908.10922
openalex publication_date 2019/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A lack of software reproducibility has become increasingly apparent in the\nlast several years, calling into question the validity of scientific findings\naffected by published tools. Reproducibility issues may have numerous sources\nof error, including the underlying numerical stability of algorithms and\nimplementations employed. Various forms of instability have been observed in\nneuroimaging, including across operating system versions, minor noise\ninjections, and implementation of theoretically equivalent algorithms. In this\npaper we explore the effect of various perturbation methods on a typical\nneuroimaging pipeline through the use of i) targeted noise injections, ii)\nMonte Carlo Arithmetic, and iii) varying operating systems to identify the\nquality and severity of their impact. The work presented here demonstrates that\neven low order computational models such as the connectome estimation pipeline\nthat we used are susceptible to noise. This suggests that stability is a\nrelevant axis upon which tools should be compared, developed, or improved,\nalongside more commonly considered axes such as accuracy/biological feasibility\nor performance. The heterogeneity observed across participants clearly\nillustrates that stability is a property of not just the data or tools\nindependently, but their interaction. Characterization of stability should\ntherefore be evaluated for specific analyses and performed on a representative\nset of subjects for consideration in subsequent statistical testing.\nAdditionally, identifying how this relationship scales to higher-order models\nis an exciting next step which will be explored. Finally, the joint application\nof perturbation methods with post-processing approaches such as bagging or\nsignal normalization may lead to the development of more numerically stable\nanalyses while maintaining sensitivity to meaningful variation.\n