Game-theoretic statistics and safe anytime-valid inference

2022/10/04 by Aaditya Ramdas, Ramdas, Aaditya, Peter Grünwald +5 · 4 voices · 20 citations
#math.ST #cs.GT #cs.IT #stat.ME

paper · pdf · doi:10.48550/arxiv.2210.01948

Abstract

Safe anytime-valid inference (SAVI) provides measures of statistical evidence and certainty -- e-processes for testing and confidence sequences for estimation -- that remain valid at all stopping times, accommodating continuous monitoring and analysis of accumulating data and optional stopping or continuation for any reason. These measures crucially rely on test martingales, which are nonnegative martingales starting at one. Since a test martingale is the wealth process of a player in a betting game, SAVI centrally employs game-theoretic intuition, language and mathematics. We summarize the SAVI goals and philosophy, and report recent advances in testing composite hypotheses and estimating functionals in nonparametric settings.

Cited by

• Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections
• Optimal Anytime-Valid Tests for Composite Nulls
• Rao-Blackwellized e-variables
• Stopping Rules for Stochastic Gradient Descent via Anytime-Valid Confidence Sequences
• Eventually LIL Regret: Almost Sure lnln T Regret for a sub-Gaussian Mixture on Unbounded Data
• A simple geometric proof for the characterisation of e-merging functions
• Ideal Attribution and Faithful Watermarks for Language Models
• E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing
• Choosing the nominal level post-hoc with knockoffs using e-values
• Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search
• Standard and comparative e-backtests for general risk measures
• Vector-valued self-normalized concentration inequalities beyond sub-Gaussianity
• When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
• Differentially Private E-Values
• Extending Prediction-Powered Inference through Conformal Prediction
• Valid Stopping for LLM Generation via Empirical Dynamic Formal Lift
• Technical note on Sequential Test-Time Adaptation via Martingale-Driven Fisher Prompting
• Sequential Test for Practical Significance: Truncated Mixture Sequential Probability Ratio Test
• Testing maximum entropy models with e-values
• Quantum Sequential Universal Hypothesis Testing

Discussions

• It's not okay, and this is basically the reason. If you want to be able to peek at the results, then increase the sample size, and repeat, you have to use anytime-valid inference techniques e.g. arxiv [bsky, 9 points, 1 comments]
• I found this paper provides a nice overview (linked from Wikipedia): Ramdas et al. 2022. Game-theoretic statistics and safe anytime valid inference. arxiv.org/abs/2210.01948 If you know any more res [bsky, 3 points, 2 comments]
• arxiv.org/abs/2210.01948 'Game-theoretic statistics and safe anytime-valid inference' - Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, Glenn Shafer How can ideas from betting and competition be used a [bsky, 2 points, 1 comments]
• expected log wealth after n rounds), and shows that the log-utility formulation characterizes the only strategy that both avoids eventual ruin (with prob 1) and achieves maximal asymptotic growth. Als [bsky, 0 points, 1 comments]

Related

• Game-Theoretic Statistics and Safe Anytime-Valid Inference
• Two ways game-theoretic probability can improve data analysis
• Li, Li, and Dai's Contribution to the Discussion of "Estimating Means of Bounded Random Variables by Betting" by Waudby-Smith and Aaditya Ramdas
• Admissible anytime-valid sequential inference must rely on nonnegative martingales
• Nonparametric Two-Sample Testing by Betting