Hypothesis Tests under Separation

2023/09/07 by Carlisle Rainey · 1 voice
Social Sciences · #Electoral Systems and Political Participation #Qualitative Comparative Analysis Research #Political Conflict and Governance

paper · pdf · doi:10.1017/pan.2023.28

openalex publication_date 2023/09/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/05/21

Abstract

Abstract Separation commonly occurs in political science, usually when a binary explanatory variable perfectly predicts a binary outcome. In these situations, methodologists often recommend penalized maximum likelihood or Bayesian estimation. But researchers might struggle to identify an appropriate penalty or prior distribution. Fortunately, I show that researchers can easily test hypotheses about the model coefficients with standard frequentist tools. While the popular Wald test produces misleading (even nonsensical) p -values under separation, I show that likelihood ratio tests and score tests behave in the usual manner. Therefore, researchers can produce meaningful p -values with standard frequentist tools under separation without the use of penalties or prior information .

Citations

• A solution to the problem of separation in logistic regression
• The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses
• Making the Most of Statistical Analyses: Improving Interpretation and Presentation

Discussions

• "Hypothesis Tests Under Separation" is now published in Political Analysis. DOI: doi.org/10.1017/pan.... If you're teaching separation in logistic regression models and/or Wald, likelihood ratio, an [bsky, 5 points, 0 comments]

Related

• Hypothesis Tests Under Separation
• Generalizations related to hypothesis testing with the Posterior distribution of the Likelihood Ratio
• The Separation Plot: A New Visual Method for Evaluating the Fit of Binary Models
• Separation in Logistic Regression: Causes, Consequences, and Control
• Robust Hypothesis Testing with Abstention