Why Propensity Scores Should Not Be Used for Matching

2019/05/07 by Gary King, Richard Nielsen, Richard A. Nielsen · 34 citations
Mathematics · #Advanced Causal Inference Techniques #Statistical Methods and Bayesian Inference #Statistical Methods and Inference

paper · pdf · doi:10.1017/pan.2019.11

openalex publication_date 2019/05/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

We show that propensity score matching (PSM), an enormously popular method of preprocessing data for causal inference, often accomplishes the opposite of its intended goal—thus increasing imbalance, inefficiency, model dependence, and bias. The weakness of PSM comes from its attempts to approximate a completely randomized experiment, rather than, as with other matching methods, a more efficient fully blocked randomized experiment. PSM is thus uniquely blind to the often large portion of imbalance that can be eliminated by approximating full blocking with other matching methods. Moreover, in data balanced enough to approximate complete randomization, either to begin with or after pruning some observations, PSM approximates random matching which, we show, increases imbalance even relative to the original data. Although these results suggest researchers replace PSM with one of the other available matching methods, propensity scores have other productive uses.

Citations

• A Theorem on the Manipulability of Redundant Serial Kinematic Chains
• Some Methods of Propensity‐Score Matching had Superior Performance to Others: Results of an Empirical Investigation and Monte Carlo simulations
• Comparison of Multivariate Matching Methods: Structures, Distances, and Algorithms
• Inference and missing data
• Dealing with limited overlap in estimation of average treatment effects
• The Dangers of Extreme Counterfactuals
• Estimating causal effects of treatments in randomized and nonrandomized studies.
• Why Most Published Research Findings Are False
• The central role of the propensity score in observational studies for causal effects
• SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
• False-Positive Psychology
• Variable Selection for Propensity Score Models
• Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score

Cited by

• A study on the impact of open doors: investigating the influence of short leave and open prison on recidivism rates
• The prosecutor's cut: Body‐worn camera evidence and criminal case dispositions
• Outcomes in Early Adulthood for Individuals Born Very Preterm and/or with Very Low Birth Weight: Evidence from Multinational Cohorts
• Violence against civil society actors in democracies: Territorialization of criminal economies and the assassination of social activists in Brazil
• The relative and joint effects of gunshot detection technology and video surveillance cameras on case clearance in Chicago
• How measurement affects causal inference: Attenuation bias is (usually) more important than outcome scoring weights
• A deterministic matching method for exact matchings to compare the outcome of different interventions
• How to Reform Without Reforming: School District Racial Composition and Pennsylvania’s “Fair” Funding Formula
• How does a focusing event shape public opinion? Natural experimental evidence from the Orlando mass shooting
• The labour market differences between the asylum population and other migrants
• Organic cultivation and farmland prices: Does certification matter?
• Evaluating Impact Using Time-Series Data
• Core concepts in statistics and research methods. Part 2: clinical research principles and observational studies
• Estimating Causal Effects With Observational Data: Guidelines for Agricultural and Applied Economists
• Black Troops, White Rage, and Political Violence in the Postbellum American South
• Comparing elite and citizen attitudes towards the differentiated implementation of EU law: evidence from a large- N survey of citizens, politicians and bureaucrats
• How the internet and social media reduce government approval: empirical evidence from Russian regions
• Good for the goose, bad for the gander? Corruption and income inequality
• Cultural‐Policy Framework and Mothers' Earnings Penalty: A European Comparison
• Sharp as a Fox: Are foxnews.com Visitors Less Politically Knowledgeable?
• MultiMBNN: Matched and Balanced Causal Inference with Neural Networks
• Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models
• Evaluating sources of bias in observational studies of angiotensin-converting enzyme inhibitor/angiotensin II receptor blocker use during COVID-19: beyond confounding
• The Efficacy of Propensity Score Matching for Separating Selection and Measurement Effects Across Different Survey Modes
• A Python Library For Empirical Calibration
• Who Does the Caring? Gender Disparities in COVID-19 Attitudes and Behaviors
• The Representational Consequences of Electronic Voting Reform
• Personality traits of special forces operators: Comparing commandos, candidates, and controls.
• The impact of health shocks on worker performance: Evidence from Major League Baseball
• Price spikes, temporary price caps, and welfare effects of regulatory interventions on wholesale electricity markets
• Bias and High-Dimensional Adjustment in Observational Studies of Peer Effects
• The effects of EU-funded enterprise grants on firms and workers
• Spanking and children's early academic skills: Strengthening causal estimates
• A Two-Stage Interpretable Matching Framework for Causal Inference
• Experiment [wikipedia]

Related

• Propensity Score Matching: Should We Use It in Designing Observational Studies?
• Propensity Score Matching underestimates Real Treatment Effect, in a simulated theoretical multivariate model
• A deterministic balancing score algorithm to avoid common pitfalls of propensity score matching
• Tuning-parameter-free optimal propensity score matching approach for causal inference
• On propensity score matching with a diverging number of matches