Ordered Risk Minimization: Learning More from Less Data

2023/03/16 by Peter Coppens, Panagiotis Patrinos, Coppens, Peter +1 · 1 citation
Decision Sciences · Economics, Econometrics and Finance · Mathematics · #FOS: Mathematics #Health Systems, Economic Evaluations, Quality of Life #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2303.09196

openalex publication_date 2023/03/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

We consider the worst-case expectation of a permutation invariant ambiguity set of discrete distributions as a proxy-cost for data-driven expected risk minimization. For this framework, we coin the term ordered risk minimization to highlight how results from order statistics inspired the proxy-cost. Specifically, we show how such costs serve as point-wise high-confidence upper bounds of the expected risk. The confidence level can be determined tightly for any sample size. Conversely we also illustrate how to calibrate the size of the ambiguity set such that the high-confidence upper bound has some user specified confidence. This calibration procedure notably supports ϕ-divergence based ambiguity sets. Numerical experiments then illustrate how the resulting scheme both generalizes better and is less sensitive to tuning parameters compared to the empirical risk minimization approach.

Cited by

• Bayesian distributionally robust variational inequalities: regularization and quantification

Related

• A Distribution Optimization Framework for Confidence Bounds of Risk Measures
• Empirical or Invariant Risk Minimization? A Sample Complexity\n Perspective
• On the Trade-Off Between Distributional Belief and Ambiguity: Conservatism, Finite-Sample Guarantees, and Asymptotic Properties
• Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimization
• Structured ambiguity sets for distributionally robust optimization