Separation in Logistic Regression: Causes, Consequences, and Control
2017/08/15 by Mohammad Ali Mansournia, Mohammad Alì Mansournia, Angelika Geroldinger +2 · 33 citations
Mathematics · #Statistical Methods in Epidemiology #Statistical Methods and Bayesian Inference #Advanced Statistical Methods and Models
paper · pdf · doi:10.1093/aje/kwx299
Abstract
Separation is encountered in regression models with a discrete outcome (such as logistic regression) where the covariates perfectly predict the outcome. It is most frequent under the same conditions that lead to small-sample and sparse-data bias, such as presence of a rare outcome, rare exposures, highly correlated covariates, or covariates with strong effects. In theory, separation will produce infinite estimates for some coefficients. In practice, however, separation may be unnoticed or mishandled because of software limits in recognizing and handling the problem and in notifying the user. We discuss causes of separation in logistic regression and describe how common software packages deal with it. We then describe methods that remove separation, focusing on the same penalized-likelihood techniques used to address more general sparse-data problems. These methods improve accuracy, avoid software problems, and allow interpretation as Bayesian analyses with weakly informative priors. We discuss likelihood penalties, including some that can be implemented easily with any software package, and their relative advantages and disadvantages. We provide an illustration of ideas and methods using data from a case-control study of contraceptive practices and urinary tract infection.
Citations
Cited by
- Choosing informative priors in Bayesian regression models: a simulation study and tutorial using Stan and R
- Firth‐Type Penalized Methods of the Modified Poisson and Least‐Squares Regression Analyses for Binary Outcomes
- Eucalyptus cover as the primary driver of native forest bird reductions: Evidence from a stand-scale analysis in NW Iberia
- Phases of methodological research in biostatistics—Building the evidence base for new methods
- Declination and Segmentation in Children with Childhood Apraxia of Speech
- Oxidative stress: a sex-specific cost of parental care
- Tuning in ridge logistic regression to solve separation
- Network meta‐analysis of rare events using penalized likelihood regression
- Case-control matching: effects, misconceptions, and recommendations. [europepmc]
- Insulin Resistance Associated With Differentiated Thyroid Carcinoma: Penalized Conditional Logistic Regression Analysis of a Matched Case-Control Study Data. [europepmc]
- Evaluating large-scale propensity score performance through real-world and synthetic data experiments. [europepmc]
- Ambulatory blood pressure monitoring and diabetes complications: Targeting morning blood pressure surge and nocturnal dipping. [europepmc]
- Time-to-event analysis for sports injury research part 2: time-varying outcomes. [europepmc]
- Penalized logistic regression with low prevalence exposures beyond high dimensional settings. [europepmc]
- Community-Based Antiretroviral Therapy (ART) Delivery for Female Sex Workers in Tanzania: 6-Month ART Initiation and Adherence. [europepmc]
- Keep calm and carry on testing: a substantive reanalysis and critique of 'what is the evidence for and validity of return-to-sport testing after anterior cruciate ligament reconstruction surgery? A systematic review and meta-analysis'. [europepmc]
- Bring More Data!-A Good Advice? Removing Separation in Logistic Regression by Increasing Sample Size. [europepmc]
- The prevalence and predictive factors of breast cancer screening among older Ghanaian women. [europepmc]
- An advanced prediction model for postoperative complications and early implant failure. [europepmc]
- Immunomodulatory sphingosine-1-phosphates as plasma biomarkers of Alzheimer's disease and vascular cognitive impairment. [europepmc]
- IL-6 Inhibition in Critically Ill COVID-19 Patients Is Associated With Increased Secondary Infections. [europepmc]
- A CHecklist for statistical Assessment of Medical Papers (the CHAMP statement): explanation and elaboration. [europepmc]
- Bias in Odds Ratios From Logistic Regression Methods With Sparse Data Sets. [europepmc]
- To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. [europepmc]
- Association between maternal thyroid function and risk of gestational hypertension and pre-eclampsia: a systematic review and individual-participant data meta-analysis. [europepmc]
- Application and interpretation of deep learning methods for the geographical origin identification of Radix Glycyrrhizae using hyperspectral imaging. [europepmc]
- Phases of methodological research in biostatistics-Building the evidence base for new methods. [europepmc]
- Non-falciparum malaria infection and IgG seroprevalence among children under 15 years in Nigeria, 2018. [europepmc]
- Nadir creatinine as a predictor of renal outcomes in PUVs: A systematic review and meta-analysis. [europepmc]
- Co-occurrence of depression, anxiety, and perinatal posttraumatic stress in postpartum persons. [europepmc]
- Leave-one-out cross-validation, penalization, and differential bias of some prediction model performance measures-a simulation study. [europepmc]
- The ADNI4 Digital Study: A novel approach to recruitment, screening, and assessment of participants for AD clinical research. [europepmc]
- Geographic and age variations in mutational processes in colorectal cancer. [europepmc]
Related