2024/12/23 by Andrea Saltelli, Alessio Lachi, Arnald Puy +1 · 1 voice · 1 citation
Decision Sciences · Materials Science · Physics and Astronomy · #Probabilistic and Robust Engineering Design #Machine Learning in Materials Science #Model Reduction and Neural Networks
paper · pdf · doi:10.31222/osf.io/sq34n
openalex publication_date 2024/12/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/14
We look at the issue of “analytic flexibility” discovered in the context of recent multianalyststudies, and reconnect it to procedures long advocated across disciplines to test thequality of a quantification. In particular, we recall Leamer’s 1985 work suggesting globalsensitivity analysis (GSA) to test the robustness of a quantitative inference, and illustratehow GSA has been successfully applied to this effect in mathematical modelling in the pastdecades. We show how this approach permits analysts to properly chart statistical gardensof forking paths before venturing into one, or to make sense of a multi-analyst experimentafter it has been done. GSA offers procedures for efficient exploration of the multidimensionalspace scanned by the modelling choices, and easily allows the thus-far unexplained“universe of uncertainty” hidden in multi-analyst studies to be unveiled (uncertainty quantification)and characterised (sensitivity analysis). We describe some essential elementsfrom the GSA toolbox and illustrate their application to a recent multi-analyst study fromBreznau, Rinke and Wuttke (2022). We call our application of GSA to the garden of theforking path a “modelling of the modelling process” (MOMP), detailing the differencesbetween this and a more recently proposed “multiverse analysis”. We conclude offering aprogramming environment for these studies.