2024/09/11 by Thomas Ball, Shuo Chen, Ball, Thomas +3 · 6 citations
Economics, Econometrics and Finance · Mathematics · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Credit Risk and Financial Regulations #Diverse Scientific and Economic Studies #FOS: Computer and information sciences #Machine Learning (cs.LG) #Probability and Statistical Research
paper · pdf · doi:10.48550/arxiv.2409.07638
openalex publication_date 2024/09/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper we explore evaluation of LLM capabilities. We present measurements of GPT-4 performance on several deterministic tasks; each task involves a basic calculation and takes as input parameter some element drawn from a large well-defined population (e.g., count elements in a list, multiply two k-digit numbers, etc). We examine several conditions per-task and perform enough trials so that statistically significant differences can be detected. This allows us to investigate the sensitivity of task-accuracy both to query phrasing and input parameter population. We find that seemingly trivial modifications in the task-prompt or input population can yield differences far larger than can be explained by sampling effects. For example, performance on a simple list-counting task varies with query-phrasing and list-length, but also with list composition (i.e., the thing-to-be-counted) and object frequency (e.g., success when an element accounts for ≈ 50% of a list is different from when it accounts for ≈ 70% etc). We conclude that efforts to quantify LLM capabilities easily succumb to the language-as-fixed-effect fallacy, where experimental observations are improperly generalized beyond what the data supports. A consequence appears to be that intuitions that have been formed based on interactions with humans form a very unreliable guide as to which input modifications should ``make no difference'' to LLM performance.