Prediction Policy Problems

2015/05/01 by Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan +1 · 585 citations
Economics, Econometrics and Finance · Mathematics · #Advanced Causal Inference Techniques #Artificial intelligence #Causal inference #Computer science #Econometrics #Economics #Field (mathematics) #Health Systems, Economic Evaluations, Quality of Life #Healthcare Policy and Management #Inference #Machine learning #Mathematics #Regression #Simple (philosophy) #Statistics #Welfare

paper · open access · doi:10.1257/aer.p20151023

published in American Economic Review 105(5), 491-495 (American Economic Association)

openalex publication_date 2015/05/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

Most empirical policy work focuses on causal inference. We argue an important class of policy problems does not require causal inference but instead requires predictive inference. Solving these “prediction policy problems” requires more than simple regression techniques, since these are tuned to generating unbiased estimates of coefficients rather than minimizing prediction error. We argue that new developments in the field of “machine learning” are particularly useful for addressing these prediction problems. We use an example from health policy to illustrate the large potential social welfare gains from improved prediction.

Cited by

• Productivity and Selection of Human Capital with Machine Learning
• Estimating treatment effects with machine learning
• Transparency, Reproducibility, and the Credibility of Economics Research
• The origins of unpredictability in life outcome prediction tasks
• Idea2Plan: Exploring AI-Powered Research Planning
• Explanations are a Means to an End: Decision Theoretic Explanation Evaluation
• Complementarities between algorithmic and human decision-making: The case of antibiotic prescribing
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Optimal Comprehensible Targeting
• A Causal Machine Learning Framework for Predicting Preventable Hospital Readmissions
• Algorithm Supported Induction for Building Theory: How Can We Use Prediction Models to Theorize?
• Applying machine learning to identify unrecognized COVID-19 deaths recorded as other causes of death in the United States
• Performative Learning Theory
• Machine-Learning Tests for Effects on Multiple Outcomes
• Forecasting House Prices
• The Flaws of Policies Requiring Human Oversight of Government Algorithms
• Machine learning who to nudge: Causal vs predictive targeting in a field experiment on student financial aid renewal
• Feedback Effects in Repeat-Use Criminal Risk Assessments
• Simple Rules to Guide Expert Classifications
• Individual Explanations in Machine Learning Models: A Case Study on Poverty Estimation
• Quantum Inspired Legal Tech Environmental Integration for Emergency Pharmaceutical Logistics with Entropy Modulated Collapse and Multilevel Governance
• Financial Regulation and AI: A Faustian Bargain?
• Measuring the predictability of life outcomes with a scientific mass collaboration
• Machine Learning: An Applied Econometric Approach
• The impact of entrepreneurship orientation on project performance: A machine learning approach
• Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment
• How to Learn when Data Reacts to Your Model: Performative Gradient Descent
• Predicting policy funding allocation with Machine Learning
• Optimal Targeting in Fundraising: A Causal Machine-Learning Approach
• A Pipeline for Variable Selection and False Discovery Rate Control With an Application in Labor Economics
• A Hierarchy of Limitations in Machine Learning
• Predictive Multiplicity in Classification
• Fine-grained prediction of food insecurity using news streams
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• On monitoring development indicators using high resolution satellite images
• Individual Explanations in Machine Learning Models: A Survey for Practitioners
• Identifying Politically Connected Firms: A Machine Learning Approach*
• Performance metrics for intervention-triggering prediction models do not reflect an expected reduction in outcomes from using the model
• Satisficing: Integrating Two Traditions
• Randomization bias in field trials to evaluate targeting methods
• Predicting the Future - Big Data, Machine Learning, and Clinical Medicine. [europepmc]
• Does Machine Learning Automate Moral Hazard and Error? [europepmc]
• HUMAN DECISIONS AND MACHINE PREDICTIONS. [europepmc]
• Machine learning in population health: Opportunities and threats. [europepmc]
• Algorithmic Accountability and Public Reason. [europepmc]
• Estimating treatment effects with machine learning. [europepmc]
• Evaluation of machine learning methodology for the prediction of healthcare resource utilization and healthcare costs in patients with critical limb ischemia-is preventive and personalized approach on the horizon? [europepmc]
• Measuring the predictability of life outcomes with a scientific mass collaboration. [europepmc]
• Validation of a Machine Learning Model to Predict Childhood Lead Poisoning. [europepmc]
• Local mortality estimates during the COVID-19 pandemic in Italy. [europepmc]
• Prescription Opioid Laws and Opioid Dispensing in US Counties: Identifying Salient Law Provisions With Machine Learning. [europepmc]
• The Fermi-Dirac distribution provides a calibrated probabilistic output for binary classifiers. [europepmc]
• Is the Way Forward to Step Back? Documenting the Frequency With Which Study Goals Are Misaligned With Study Methods and Interpretations in the Epidemiologic Literature. [europepmc]
• Predicting avoidable hospital events in Maryland. [europepmc]
• Machine learning models for 180-day mortality prediction of patients with advanced cancer using patient-reported symptom data. [europepmc]
• Researcher reasoning meets computational capacity: Machine learning for social science. [europepmc]
• Prediction model for myocardial injury after non-cardiac surgery using machine learning. [europepmc]
• Does Automatic Wage Indexation Destroy Jobs? A Machine Learning Approach. [europepmc]
• Predicting food crises using news streams. [europepmc]
• Determinants of people's motivations to approach COVID-19 vaccination centers. [europepmc]
• Introduction to the Special Collection on the Fragile Families Challenge. [europepmc]
• Privacy, Ethics, and Data Access: A Case Study of the Fragile Families Challenge. [europepmc]
• The quest for the reliability of machine learning models in binary classification on tabular data. [europepmc]
• The origins of unpredictability in life outcome prediction tasks. [europepmc]
• DIAGNOSING PHYSICIAN ERROR: A MACHINE LEARNING APPROACH TO LOW-VALUE HEALTH CARE. [europepmc]

Related

• The Value of Prediction in Identifying the Worst-Off
• The Relative Value of Prediction in Algorithmic Decision Making
• Machine learning in policy evaluation: new tools for causal inference
• When the whole is greater than the sum of its parts: why machine learning and conventional statistics are complementary for predicting future health outcomes
• Policy Learning with Rare Outcomes