Scalable logistic regression with crossed random effects

2021/05/28 by Swarnadip Ghosh, Trevor Hastie, Ghosh, Swarnadip +3 · 3 citations
Computer Science · Mathematics · Physics and Astronomy · #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Methodology (stat.ME) #Statistical Mechanics and Entropy #Statistical Methods and Bayesian Inference #Statistics Theory (math.ST) #math.ST #stat.CO #stat.ME #stat.TH

paper · pdf · doi:10.48550/arxiv.2105.13747

32 pages, 5 figures

openalex publication_date 2021/05/28 · arxiv created 2021/12/29 · arxiv updated 2021/12/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The cost of both generalized least squares (GLS) and Gibbs sampling in a crossed random effects model can easily grow faster than N^3/2 for N observations. Ghosh et al. (2020) develop a backfitting algorithm that reduces the cost to O(N). Here we extend that method to a generalized linear mixed model for logistic regression. We use backfitting within an iteratively reweighted penalized least square algorithm. The specific approach is a version of penalized quasi-likelihood due to Schall (1991). A straightforward version of Schall's algorithm would also cost more than N^3/2 because it requires the trace of the inverse of a large matrix. We approximate that quantity at cost O(N) and prove that this substitution makes an asymptotically negligible difference. Our backfitting algorithm also collapses the fixed effect with one random effect at a time in a way that is analogous to the collapsed Gibbs sampler of Papaspiliopoulos et al. (2020). We use a symmetric operator that facilitates efficient covariance computation. We illustrate our method on a real dataset from Stitch Fix. By properly accounting for crossed random effects we show that a naive logistic regression could underestimate sampling variances by several hundred fold.

Cited by

• Universal inference for variance components
• A Linear Approach to Data Poisoning
• Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects

Related

• Backfitting for large scale crossed random effects regressions
• Scalable inference for crossed random effects models
• Linear-cost unbiased posterior estimates for crossed effects and matrix factorization models via couplings
• Precise Asymptotics for Linear Mixed Models with Crossed Random Effects
• Scalable solution to crossed random effects model with random slopes