Firth's logistic regression with rare events: accurate effect estimates and predictions?

2017/03/12 by Rainer Puhr, Georg Heinze, Mariana Nold +2 · 1 voice · 8 citations
Environmental Science · Mathematics · #Fuzzy Systems and Optimization #Hydrology and Drought Analysis #Statistical Methods in Epidemiology

paper · doi:10.1002/sim.7273

openalex publication_date 2017/03/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Firth's logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards one-half is introduced in the predicted probabilities. The stronger the imbalance of the outcome, the more severe is the bias in the predicted probabilities. We propose two simple modifications of Firth's logistic regression resulting in unbiased predicted probabilities. The first corrects the predicted probabilities by a post hoc adjustment of the intercept. The other is based on an alternative formulation of Firth's penalization as an iterative data augmentation procedure. Our suggested modification consists in introducing an indicator variable that distinguishes between original and pseudo-observations in the augmented data. In a comprehensive simulation study, these approaches are compared with other attempts to improve predictions based on Firth's penalization and to other published penalization strategies intended for routine use. For instance, we consider a recently suggested compromise between maximum likelihood and Firth's logistic regression. Simulation results are scrutinized with regard to prediction and effect estimation. We find that both our suggested methods do not only give unbiased predicted probabilities but also improve the accuracy conditional on explanatory variables compared with Firth's penalization. While one method results in effect estimates identical to those of Firth's penalization, the other introduces some bias, but this is compensated by a decrease in the mean squared error. Finally, all methods considered are illustrated and compared for a study on arterial closure devices in minimally invasive cardiac surgery. Copyright © 2017 John Wiley & Sons, Ltd.

Citations

• A solution to the problem of separation in logistic regression
• Bias reduction of maximum likelihood estimates

Cited by

• Surrogate-Powered Inference: Regularization and Adaptivity
• The role of cytonuclear interactions to plant adaptation across a <i>Populus</i> hybrid zone
• Phases of methodological research in biostatistics—Building the evidence base for new methods
• Evidence for toxin-encoding coinfections driving intransitive dynamics between allelopathic phenotypes in natural yeast populations
• Debts and experienced financial scarcity: associations with nonsuicidal self‐injury and suicidality in adolescents at risk for psychopathology
• Tuning in ridge logistic regression to solve separation
• Bias reduction in g-computation for covariate adjustment in randomized clinical trials
• Learning Theory and Algorithms for Revenue Management in Sponsored Search

Discussions

• Same, often with the FLAC modification. Sadly, the `logistf` package does not play entirely well with other packages (mice, broom). Puhr et al. (2017). Firth’s logistic regression with rare events: ac [bsky, 2 points, 1 comments]

Related

• A simulation study of the number of events per variable in logistic regression analysis
• Inferential tools in penalized logistic regression for small and sparse data: A comparative study
• Jeffreys-prior penalty for high-dimensional logistic regression: A conjecture about aggregate bias
• Logistic Regression: A Brief Primer
• Correspondence: The use of cost information when defining critical values for prediction of rare events using logistic regression and similar methods