Score-based calibration testing for multivariate forecast distributions

2022/11/29 by Knüppel, Malte, Krüger, Fabian, Pohle, Marc-Oliver · 1 citation
#Econometrics (econ.EM) #FOS: Computer and information sciences #FOS: Economics and business #Methodology (stat.ME)

paper · doi:10.48550/arxiv.2211.16362

Abstract

Calibration tests based on the probability integral transform (PIT) are routinely used to assess the quality of univariate distributional forecasts. However, PIT-based calibration tests for multivariate distributional forecasts face various challenges. We propose two new types of tests based on proper scoring rules, which overcome these challenges. They arise from a general framework for calibration testing in the multivariate case, introduced in this work. The new tests have good size and power properties in simulations and solve various problems of existing tests. We apply the tests to forecast distributions for macroeconomic and financial time series data.

Cited by

• A Composite-Loss Graph Neural Network for the Multivariate Post-Processing of Ensemble Weather Forecasts

Related

• Evaluating the Discrimination Ability of Proper Multivariate Scoring Rules
• Calibrated Multivariate Regression with Localized PIT Mappings
• Multivariate Forecasting Evaluation: On Sensitive and Strictly Proper Scoring Rules
• Calibration by Distribution Matching: Trainable Kernel Calibration Metrics
• Scores for Multivariate Distributions and Level Sets