A New Vector Partition of the Probability Score
1973/06/01 by Allan H. Murphy · 1,095 citations
Decision Sciences · Mathematics · #Combinatorics #Computer science #Data mining #Forecasting Techniques and Applications #Mathematics #Measure (data warehouse) #Partition (number theory) #Statistics
paper · pdf · doi:10.1175/1520-0450(1973)012<0595:anvpot>2.0.co;2
published in Journal of applied meteorology 12(4), 595-600 (American Meteorological Society)
openalex publication_date 1973/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04
Abstract
A new vector partition of the probability, or Brier, score (PS) is formulated and the nature and properties of this partition are described. The relationships between the terms in this partition and the terms in the original vector partition of the PS are indicated. The new partition consists of three terms: 1) a measure of the uncertainty inherent in the events, or states, on the occasions of concern (namely, the PS for the sample relative frequencies); 2) a measure of the reliability of the forecasts; and 3) a new measure of the resolution of the forecasts. These measures of reliability and resolution are and are not, respectively, equivalent (i.e., linearly related) to the measures of reliability and resolution provided by the original partition. Two sample collections of probability forecasts are used to illustrate the differences and relationships between these partitions. Finally, the two partitions are compared, with particular reference to the attributes of the forecasts with which the partitions are concerned, the interpretation of the partitions in geometric terms, and the use of the partitions as the bases for the formulation of measures to evaluate probability forecasts. The results of these comparisons indicate that the new partition offers certain advantages vis-à-vis the original partition.
Cited by
- Two-stage dynamic signal detection: A theory of choice, decision time, and confidence.
- The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossible
- Rethinking Early Stopping: Refine, Then Calibrate
- Bellman Calibration for V-Learning in Offline Reinforcement Learning
- Brenier Isotonic Regression
- Bayesian Complete-Pooling in Cross-Subject Classification for Motor Imagery Electroencephalogram
- Spline-Based Probability Calibration
- Predicting Lockean from gradational accuracy
- HydroDiffusion: Diffusion-Based Probabilistic Streamflow Forecasting with a State Space Backbone
- Improving Multi-Class Calibration through Normalization-Aware Isotonic Techniques
- Are we misdiagnosing ensemble forecast reliability? On the insufficiency of Spread-Error and rank-based reliability metrics
- Enriching Psychological Research by Exploring the Source and Nature of Noise
- More on verification of probability forecasts for football outcomes: score decompositions, reliability, and discrimination analyses
- Flare Forecasting Using the Evolution of McIntosh Sunspot Classifications
- Universal Inference for Testing Calibration of Mean Estimates within the Exponential Dispersion Family
- Stable Discovery of Interpretable Subgroups via Calibration in Causal Studies
- Right Decisions from Wrong Predictions: A Mechanism Design Alternative to Individual Calibration
- Capturing Intransitive Dominance in Tennis Forecasting: A Graph Neural Network Approach
- Calibration and Discrimination Optimization Using Clusters of Learned Representation
- Individual differences in reasoning: Implications for the rationality debate?
- Memory metaphors and the real-life/laboratory controversy: Correspondence versus storehouse conceptions of memory
- Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
- Model Monitoring: A General Framework with an Application to Non-life Insurance Pricing
- Understanding forecast verification statistics
- Memory compression and physical state augmentation favor different AMOC prediction tasks
- Gaussian Process Modeling with Genotype x Environment Kernels for Wheat Performance Prediction
- Who has the best probabilities? Luck versus skill in prediction tournaments
- Dynamic Prediction of Joint Longitudinal-Survival Models Using a Similarity-Based Approach
- Assessing the conditional calibration of interval forecasts using decompositions of the interval score
- Non-Parametric Calibration for Classification
- Gender Differences in the Self-Assessment of Accuracy on Cognitive Tasks
- Beyond Accuracy: How AI Metacognitive Sensitivity improves AI-assisted Decision Making
- Adaptive Label Smoothing
- On Deep Neural Network Calibration by Regularization and its Impact on Refinement
- Threshold Choice Methods: the Missing Link
- Large-scale probabilistic predictors with and without guarantees of validity
- The Role of Individual Differences in the Accuracy of Confidence Judgments
- Stable discovery of interpretable subgroups via calibration in causal studies
- An analysis of machine learning approaches for enhancing decision-making in complex discrete choice tasks
- Collaborative Sampling in Generative Adversarial Networks
- On Equivariant Model Selection through the Lens of Uncertainty
- h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
- Combining and Extremizing Real-Valued Forecasts
- Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs
- Enforcing tail calibration when training probabilistic forecast models
- Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks
- Earthquake Counting Method for Spatially Localized Probabilities: Challenges in Real-Time Information Delivery
- Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability
- Post-processing of wind gusts from COSMO-REA6 with a spatial Bayesian hierarchical extreme value model
- Practical estimation of the optimal classification error with soft labels and calibration
- On the Role of Dataset Quality and Heterogeneity in Model Confidence
- Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration
- Improved reliability and accuracy of CMIP5 global mean surface temperature projections
- The Manokhin Probability Matrix: A Diagnostic Framework for Classifier Probability Quality
- Understanding Misunderstanding: Social Psychological Perspectives
- Inside the Planning Fallacy: The Causes and Consequences of Optimistic Time Predictions
- Martingale Doppelgänger-Eval: An Identification Framework for Auditing Candlestick Understanding in Vision-Language Models
- Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
- Do those who know more also know more about how much they know?
- Support theory: A nonextensional representation of subjective probability.
- ForesightFlow: An Information Leakage Score Framework for Prediction Markets
- Coordination as an Architectural Layer for LLM-Based Multi-Agent Systems
- Like Goes with Like: The Role of Representativeness in Erroneous and Pseudo-Scientific Beliefs
- Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores
- On the visualization, verification and recalibration of ternary probabilistic forecasts
- Optimising HEP parameter fits via Monte Carlo weight derivative regression
- Clinical Versus Actuarial Judgment
- Heuristics and Biases
- When Predictions Fail: The Dilemma of Unrealistic Optimism
- Skill of data based predictions versus dynamical models -- case study on\n extreme temperature anomalies
- Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability.
- Sympathetic Magical Thinking: The Contagion and Similarity “Heuristics”
- A comparative analysis of predictive models of morbidity in intensive care unit after cardiac surgery - part I: model planning. [europepmc]
- Computerized prediction of intensive care unit discharge after cardiac surgery: development and validation of a Gaussian processes model. [europepmc]
- On the reliability of seasonal climate forecasts. [europepmc]
- Global forecasting of thermal health hazards: the skill of probabilistic predictions of the Universal Thermal Climate Index (UTCI). [europepmc]
- How to measure metacognition. [europepmc]
- Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning. [europepmc]
- Cancer survival analysis using semi-supervised learning method based on Cox and AFT models with L1/2 regularization. [europepmc]
- A quantitative confidence signal detection model: 2. Confidence analysis. [europepmc]
- Learning algorithms allow for improved reliability and accuracy of global mean surface temperature projections. [europepmc]
- A novel method for assessing climate change impacts in ecotron experiments. [europepmc]
- Machine learning-based analysis of [ 18 F]DCFPyL PET radiomics for risk stratification in primary prostate cancer. [europepmc]
- Mathematically aggregating experts' predictions of possible futures. [europepmc]
- Using prediction polling to harness collective intelligence for disease forecasting. [europepmc]
- Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction. [europepmc]
- Early detection of colorectal cancer by leveraging Dutch primary care consultation notes with free text embeddings. [europepmc]
- Short-lead seasonal precipitation forecast in northeastern Brazil using an ensemble of artificial neural networks. [europepmc]
- Achieving well-informed decision-making in drug discovery: a comprehensive calibration study using neural network-based structure-activity models. [europepmc]
- Characterizing the role of early life factors in machine learning-based multimorbidity risk prediction. [europepmc]
- A-calibration: assessment of prediction models for survival data under censoring. [europepmc]
- Post-learning replay of hippocampal-striatal activity is biased by reward-prediction signals. [europepmc]
- Signal Fidelity Index-aware calibration for addressing distributional shift in predictive modeling across heterogeneous real-world data. [europepmc]
- Machine learning-based approach to guide the choice between baricitinib and tocilizumab in critical COVID-19 pneumonia treatment: a retrospective cohort study. [europepmc]
- An informed monotone neural network for multi-stage chronic kidney disease prediction with integrated GFR estimation. [europepmc]
- Post-learning replay of hippocampal-striatal activity is biased by reward-prediction signals [europepmc]
Related