Firth's logistic regression with rare events: accurate effect estimates AND predictions?

2017/03/12 by Rainer Puhr, Georg Heinze, Mariana Nold +2 · 1 voice · 10 citations
Environmental Science · Mathematics · #Fuzzy Systems and Optimization #Hydrology and Drought Analysis #Statistical Methods in Epidemiology #stat.ME

paper · pdf · doi:10.1002/sim.7273

openalex publication_date 2017/03/12 · arxiv created 2021/01/19 · arxiv updated 2021/01/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Firth-type logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards 1/2 is introduced in the predicted probabilities. The stronger the imbalance of the outcome, the more severe is the bias in the predicted probabilities. We propose two simple modifications of Firth-type logistic regression resulting in unbiased predicted probabilities. The first corrects the predicted probabilities by a post-hoc adjustment of the intercept. The other is based on an alternative formulation of Firth-types estimation as an iterative data augmentation procedure. Our suggested modification consists in introducing an indicator variable which distinguishes between original and pseudo observations in the augmented data. In a comprehensive simulation study these approaches are compared to other attempts to improve predictions based on Firth-type penalization and to other published penalization strategies intended for routine use. For instance, we consider a recently suggested compromise between maximum likelihood and Firth-type logistic regression. Simulation results are scrutinized both with regard to prediction and regression coefficients. Finally, the methods considered are illustrated and compared for a study on arterial closure devices in minimally invasive cardiac surgery.

Citations

• A solution to the problem of separation in logistic regression
• Bias reduction of maximum likelihood estimates

Cited by

• Surrogate-Powered Inference: Regularization and Adaptivity
• The role of cytonuclear interactions to plant adaptation across a Populus hybrid zone
• Phases of methodological research in biostatistics—Building the evidence base for new methods
• Evidence for toxin-encoding coinfections driving intransitive dynamics between allelopathic phenotypes in natural yeast populations
• Debts and experienced financial scarcity: associations with nonsuicidal self‐injury and suicidality in adolescents at risk for psychopathology
• Tuning in ridge logistic regression to solve separation
• Bias reduction in g-computation for covariate adjustment in randomized clinical trials
• Learning Theory and Algorithms for Revenue Management in Sponsored Search
• Oversharing: The downside of data sharing in local government
• The impact of class imbalance in logistic regression models for low-default portfolios in credit risk

Discussions

• Same, often with the FLAC modification. Sadly, the `logistf` package does not play entirely well with other packages (mice, broom). Puhr et al. (2017). Firth’s logistic regression with rare events: ac [bsky, 2 points, 1 comments]

Related

• A simulation study of the number of events per variable in logistic regression analysis
• Inferential tools in penalized logistic regression for small and sparse data: A comparative study
• Jeffreys-prior penalty for high-dimensional logistic regression: A conjecture about aggregate bias
• Logistic Regression: A Brief Primer
• Correspondence: The use of cost information when defining critical values for prediction of rare events using logistic regression and similar methods