How to Tell When Simpler, More Unified, or LessAd HocTheories will Provide More Accurate Predictions

1994/03/01 by MALCOLM FORSTER, ELLIOTT SOBER, Elliott Sober · 658 citations
Arts and Humanities · Mathematics · #Akaike information criterion #Computer science #Econometrics #Empiricism #Epistemology #Inference #Machine learning #Mathematical economics #Mathematics #Philosophy #Philosophy and History of Science #Philosophy of science #Post hoc #Probability and Statistical Research #Realism #Scientific realism #Simplicity #Underdetermination #Unification

paper · doi:10.1093/bjps/45.1.1

published in The British Journal for the Philosophy of Science 45(1), 1-35 (Oxford University Press)

openalex publication_date 1994/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/23

Abstract

Traditional analyses of the curve fitting problem maintain that the data do not indicate what form the fitted curve should take. Rather, this issue is said to be settled by prior probabilities, by simplicity, or by a background theory. In this paper, we describe a result due to Akaike [1973], which shows how the data can underwrite an inference concerning the curve's form based on an estimate of how predictively accurate it will be. We argue that this approach throws light on the theoretical virtues of parsimoniousness, unification, and non ad hocness, on the dispute about Bayesianism, and on empiricism and scientific realism.

Cited by

• Causal Inference from Noise
• Prediction Versus Accommodation and the Risk of Overfitting
• Bahasa Indonesia version of Weight Stigma Exposure Inventory (WeSEI): Translation and validation among young adults
• Quantifying Parsimony in Structural Equation Modeling
• Multimodel Inference
• The Units of Life
• Why Simpler Computer Simulation Models Can Be Epistemically Better for Informing Decisions
• AIC and the challenge of complexity: A case study from ecology
• To Explain or to Predict?
• Prediction‐Oriented Model Selection in Partial Least Squares Path Modeling
• A Plea for History and Philosophy of Statistics and Machine Learning
• Kritik am Dogma „Simple Structure“
• The simplicity of physical laws
• Scientific Method in Practice
• On the Limits of Intransitive Coordination
• Theory-laden experimentation
• Bayes not Bust! Why Simplicity is no Problem for Bayesians
• Systematizing the theoretical virtues
• Theoretical Virtues: Do Scientists Think What Philosophers Think They Ought to Think?
• The Multiple Realizability Argument Against Reductionism
• Kullback-Leibler information as a basis for strong inference in ecological studies
• Statistical learning theory and Occam's razor: Regularization
• Predicting the Future is Like Completing a Painting: Towards a Novel Method for Time-Series Forecasting
• Predicting Wimbledon 2005 tennis results by mere player name recognition
• Modelling ecological systems in a changing world. [europepmc]
• Predictive systems ecology. [europepmc]
• The probabilistic no miracles argument. [europepmc]
• What we (should) talk about when we talk about fruitfulness. [europepmc]
• Why Should Natural Principles Be Simple? [europepmc]
• Evaluating the Performance of Inclusive Growth Based on the BP Neural Network and Machine Learning Approach. [europepmc]
• The efficiency paradox of discharge masking head loss in run-of-river hydropower generation. [europepmc]
• Comparative Evidence-Based Model Choice: A Sketch of a Theory. [europepmc]

Related

• Two- and Multi-dimensional Curve Fitting using Bayesian Inference
• How Do I Know What My Theory Predicts?
• It’s all in your head—naturalness arguments do not require aleatoric uncertainty
• Fitting theory to data in the presence of background uncertainties
• Statistical efficiency of curve fitting algorithms