Position: Evaluation Scores Are Perishable Knowledge Claims

2026/01/01 by Sankalp Gilda, Shlok Gilda
Computer Science · Decision Sciences · Social Sciences · #Delphi Technique in Research #Meta-analysis and systematic reviews #Reliability and Agreement in Measurement #cs.AI #cs.CL #cs.LG #cs.SE

paper · pdf · doi:10.18653/v1/2026.gem-main.80

published as Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM), ACL 2026, pages 1029-1035 · 7 pages, 1 figure, 1 table. Published at the Fifth Workshop on Generation, Evaluation and Metrics (GEM), ACL 2026, San Diego

openalex publication_date 2026/01/01 · openalex created_date 2026/07/02 · arxiv created 2026/07/28 · arxiv updated 2026/07/30 · openalex updated_date 2026/07/30

Abstract

Evaluation methodologies for language models increasingly combine multiple signalsautomated metrics, LLM-as-judge ratings, human assessments, and benchmark suite results.When these signals are aggregated via averaging, the resulting evaluation confidence can substantially exceed the reliability of the weakest signal: a phenomenon we call trust inflation in evaluation.We argue that evaluation scores should be treated as epistemic claims with three properties: formality (human evaluation provides stronger evidence than an automated metric), scope (a benchmark result applies to the tested distribution, not universally), and validity windows (benchmark results expire as contamination accumulates and distributions shift).Drawing on several converging research traditions-chain-of-thought analysis, possibilistic logic, and algebraic theorythat establish weakest-link aggregation as the conservative endpoint of a parameterized operator family controlled by a single pessimism parameter, and on concrete lessons from building an evaluation harness for agentic AI, we propose that evaluation results carry explicit metadata-formality tier, scope declaration, and expiration date-to make their epistemic status transparent.We illustrate the cost of mean aggregation on the public HELM leaderboard: across 54 frontier models on ten scenarios, the top-five models ranked by mean score and by weakest-link are completely disjoint.

Citations

• Reasoning Models Don't Always Say What They Think
• DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
• A Survey on LLM-as-a-Judge
• Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
• Are Fact-Checking Tools Helpful? An Exploration of the Usability of Google Fact Check
• A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
• Elo Uncovered: Robustness and Best Practices in Language Model Evaluation
• Rethinking Model Evaluation as Narrowing the Socio-Technical Gap
• Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
• Model Cards for Model Reporting
• Datasheets for Datasets
• Datasheets for datasets

Related

• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
• The simulation of judgment in LLMs
• The Epistemic Suite: A Post-Foundational Diagnostic Methodology for Assessing AI Knowledge Claims
• Large Language Models are not Fair Evaluators
• Benchmarking is Broken -- Don't Let AI be its Own Judge