Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and Best Practices for Machine Learning in Chemistry

2020/10/31 by Gaurav Vishwakarma, Aditya Sonpal, Johannes Hachmann
Biochemistry, Genetics and Molecular Biology · Chemistry · Computer Science · Engineering · Materials Science · Physics and Astronomy · #Benchmarking #Chemistry #Computational Drug Discovery Methods #Computer science #Data science #Economics #Engineering #Epistemology #Machine Learning in Materials Science #Machine learning #Management #Management science #Metabolomics and Mass Spectrometry Studies #Philosophy #Quality (philosophy) #cs.LG #physics.chem-ph

paper · pdf · doi:10.1016/j.trechm.2020.12.004

published as Trends in Chemistry, 2021, ISSN 2589-5974

openalex created_date 2020/10/08 · openalex publication_date 2021/01/12 · arxiv created 2021/01/22 · arxiv updated 2021/01/26 · openalex updated_date 2026/08/05

Abstract

This review aims to draw attention to two issues of concern when we set out to make machine learning work in the chemical and materials domain, i.e., statistical loss function metrics for the validation and benchmarking of data-derived models, and the uncertainty quantification of predictions made by them. They are often overlooked or underappreciated topics as chemists typically only have limited training in statistics. Aside from helping to assess the quality, reliability, and applicability of a given model, these metrics are also key to comparing the performance of different models and thus for developing guidelines and best practices for the successful application of machine learning in chemistry.

Citations

• A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
• The Generation of a Unique Machine Description for Chemical Structures-A Technique Developed at Chemical Abstracts Service.
• Using AUC and accuracy in evaluating learning algorithms
• The accuracy of intermittent demand estimates
• An introduction to ROC analysis
• Open Babel: An open chemical toolbox
• The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling
• Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning