Re-evaluating Evaluation

2018/06/07 by David Balduzzi, Balduzzi, David, Karl Tuyls +6 · 2 voices · 16 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Science and Game Theory (cs.GT) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.GT #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1806.02643

NIPS 2018, final version

openalex publication_date 2018/06/07 · arxiv published 2018/06/07 · arxiv created 2018/10/30 · arxiv updated 2018/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Progress in machine learning is measured by careful evaluation on problems of outstanding common interest. However, the proliferation of benchmark suites and environments, adversarial attacks, and other complications has diluted the basic evaluation model by overwhelming researchers with choices. Deliberate or accidental cherry picking is increasingly likely, and designing well-balanced evaluation suites requires increasing effort. In this paper we take a step back and propose Nash averaging. The approach builds on a detailed analysis of the algebraic structure of evaluation in two basic scenarios: agent-vs-agent and agent-vs-task. The key strength of Nash averaging is that it automatically adapts to redundancies in evaluation data, so that results are not biased by the incorporation of easy tasks or weak agents. Nash averaging thus encourages maximally inclusive evaluation -- since there is no harm (computational cost aside) from including all available tasks and agents.

Citations

• Intriguing properties of neural networks
• A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
• Gradient-based learning applied to document recognition
• "General Intelligence," Objectively Determined and Measured
• ImageNet classification with deep convolutional neural networks
• Count-Based Exploration with Neural Density Models
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

Cited by

• Active Evaluation of General Agents: Problem Definition and Comparison of Baseline Algorithms
• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia
• Which Nash Equilibrium? Solver-Dependent Selection on Zero-Sum Nash Polytopes
• Beyond the high score: Prosocial ability profiles of multi-agent populations
• Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption
• Fractals2019: Combinatorial Optimisation with Dynamic Constraint Annealing
• Scaling Scaling Laws with Board Games
• The Benchmark Lottery
• Approximate exploitability: Learning a best response in large games
• Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings
• Accelerated MM Algorithms for Ranking Scores Inference from Comparison Data
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Scorio.jl: A Julia package for ranking stochastic responses
• Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
• Nash without Numbers: A Social Choice Approach to Mixed Equilibria in Context-Ordinal Games
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility

Discussions

• [🧵6/N] A previous approach, called Nash Averaging (arxiv.org/abs/1806.02643), formulated the problem as a two-player zero-sum game where a model player maximizes performance on tasks by playing stron [bsky, 2 points, 1 comments]
• Strongly recommend starting with the original Re-Evaluating Evaluation paper by Balduzzi et al. first: arxiv.org/abs/1806.02643 [bsky, 1 points, 1 comments]

Related

• Active Evaluation of General Agents: Problem Definition and Comparison of Baseline Algorithms
• Evaluating Agents using Social Choice Theory
• Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learning
• Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation
• The Benchmark Lottery