2025/02/02 by Moritz Wolter, Wolter, Moritz, Lokesh Veeramacheneni +2
Computer Science · Decision Sciences · Medicine · #Artificial Intelligence in Healthcare and Education #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Scientific Computing and Data Management #Software Engineering (cs.SE)
paper · pdf · doi:10.48550/arxiv.2502.00902
openalex publication_date 2025/02/02 · openalex created_date 2025/02/05 · openalex updated_date 2026/07/28
While experimental reproduction remains a pillar of the scientific method, we observe that the software best practices supporting the reproduction of machine learning ( ML ) research are often undervalued or overlooked, leading both to poor reproducibility and damage to trust in the ML community. We quantify these concerns by surveying the usage of software best practices in software repositories associated with publications at major ML conferences and journals such as NeurIPS, ICML, ICLR, TMLR, and MLOSS within the last decade. We report the results of this survey that identify areas where software best practices are lacking and areas with potential for growth in the ML community. Finally, we discuss the implications and present concrete recommendations on how we, as a community, can improve reproducibility in ML research.