2024/09/23 by Izaskun Mallona, Charlotte Soneson, Mallona, Izaskun +13 · 1 citation
Biochemistry, Genetics and Molecular Biology · #Bioinformatics and Genomic Networks #FOS: Biological sciences #Gene expression and cancer classification #Genetics, Bioinformatics, and Biomedical Research #Other Quantitative Biology (q-bio.OT)
paper · pdf · doi:10.48550/arxiv.2409.15472
openalex publication_date 2024/09/23 · openalex created_date 2024/10/26 · openalex updated_date 2026/07/28
Benchmarking, which involves collecting reference datasets and demonstrating method performances, is a requirement for the development of new computational tools, but also becomes a domain of its own to achieve neutral comparisons of methods. Although a lot has been written about how to design and conduct benchmark studies, this Perspective sheds light on a wish list for a computational platform to orchestrate benchmark studies. We discuss various ideas for organizing reproducible software environments, formally defining benchmarks, orchestrating standardized workflows, and how they interface with computing infrastructure.