Optimising HEP parameter fits via Monte Carlo weight derivative regression

2020/01/01 by A. Valassi, Andrea Valassi
Computer Science · Economics, Econometrics and Finance · Environmental Science · Mathematics · Physics and Astronomy · #Algorithm #Applied mathematics #Basis (linear algebra) #Event (particle physics) #Financial Risk and Volatility Modeling #Interval (graph theory) #Mathematics #Monte Carlo method #Physics #Sensitivity (control systems) #Soil Geostatistics and Mapping #Spatial and Panel Data Analysis #Statistical physics #Statistics #acm:62F10 #cs.LG #hep-ex #msc:62F10 #physics.data-an

paper · pdf · doi:10.1051/epjconf/202024506038

published as EPJ Web of Conferences 245, 06038 (2020) · 15 pages, 1 figure, submitted to CHEP2019 proceedings in EPJ Web of Conferences; revised version addressing referee's comments

openalex publication_date 2020/01/01 · openalex created_date 2020/04/03 · arxiv created 2020/09/29 · arxiv updated 2020/11/20 · openalex updated_date 2026/08/05

Abstract

HEP event selection is traditionally considered a binary classification problem, involving the dichotomous categories of signal and background. In distribution fits for particle masses or couplings, however, signal events are not all equivalent, as the signal differential cross section has different sensitivities to the measured parameter in different regions of phase space. In this paper, I describe a mathematical framework for the evaluation and optimization of HEP parameter fits, where this sensitivity is defined on an event-by-event basis, and for MC events it is modeled in terms of their MC weight derivatives with respect to the measured parameter. Minimising the statistical error on a measurement implies the need to resolve (i.e. separate) events with different sensitivities, which ultimately represents a non-dichotomous classification problem. Since MC weight derivatives are not available for real data, the practical strategy I suggest consists in training a regressor of weight derivatives against MC events, and then using it as an optimal partitioning variable for 1-dimensional fits of data events. This CHEP2019 paper is an extension of the study presented at CHEP2018: in particular, event-by-event sensitivities allow the exact computation of the “FIP” ratio between the Fisher information obtained from an analysis and the maximum information that could possibly be obtained with an ideal detector. Using this expression, I discuss the relationship between FIP and two metrics commonly used in Meteorology (Brier score and MSE), and the importance of “sharpness” both in HEP and in that domain. I finally point out that HEP distribution fits should be optimized and evaluated using probabilistic metrics (like FIP or MSE), whereas ranking metrics (like AUC) or threshold metrics (like accuracy) are of limited relevance for these specific problems.

Citations

• Sensitivity of searches for new signals and its optimization
• IR evaluation methods for retrieving highly relevant documents
• The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
• Evaluation of three methods for calculating statistical significance when incorporating a systematic uncertainty into a test of the background-only hypothesis for a Poisson process
• A New Vector Partition of the Probability Score
• Cumulated gain-based evaluation of IR techniques
• VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
• First Observation of Elec-troweak Single Top Quark Production
• Learning from Imbalanced Data
• The use of the area under the ROC curve in the evaluation of machine learning algorithms
• The meaning and use of the area under a receiver operating characteristic (ROC) curve.
• An introduction to ROC analysis
• A systematic analysis of performance measures for classification tasks
• The Higgs Machine Learning Challenge
• Ranking and empirical minimization of U-statistics
• Classification And Regression Trees