2023/11/30 by Jake M. Hofman, Angelos Chatzimparmpas, Hofman, Jake M. +7 · 3 citations
Computer Science · Materials Science · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Machine Learning in Materials Science #Methodology (stat.ME)
paper · pdf · doi:10.48550/arxiv.2311.18807
openalex publication_date 2023/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Amid rising concerns of reproducibility and generalizability in predictive modeling, we explore the possibility and potential benefits of introducing pre-registration to the field. Despite notable advancements in predictive modeling, spanning core machine learning tasks to various scientific applications, challenges such as overlooked contextual factors, data-dependent decision-making, and unintentional re-use of test data have raised questions about the integrity of results. To address these issues, we propose adapting pre-registration practices from explanatory modeling to predictive modeling. We discuss current best practices in predictive modeling and their limitations, introduce a lightweight pre-registration template, and present a qualitative study with machine learning researchers to gain insight into the effectiveness of pre-registration in preventing biased estimates and promoting more reliable research outcomes. We conclude by exploring the scope of problems that pre-registration can address in predictive modeling and acknowledging its limitations within this context.