Logistic regression with total variation regularization

2020/03/05 by Sara van de Geer, van de Geer, Sara
Engineering · Mathematics · #62J12 62J07 #FOS: Mathematics #Sparse and Compressive Sensing Techniques #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2003.02678

openalex publication_date 2020/03/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study logistic regression with total variation penalty on the canonical parameter and show that the resulting estimator satisfies a sharp oracle inequality: the excess risk of the estimator is adaptive to the number of jumps of the underlying signal or an approximation thereof. In particular when there are finitely many jumps, and jumps up are sufficiently separated from jumps down, then the estimator converges with a parametric rate up to a logarithmic term log n / n, provided the tuning parameter is chosen appropriately of order 1/ √ n. Our results extend earlier results for quadratic loss to logistic loss. We do not assume any a priori known bounds on the canonical parameter but instead only make use of the local curvature of the theoretical risk.

Citations

• Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation

Related

• Adaptive Risk Bounds in Univariate Total Variation Denoising and Trend Filtering
• Logistic Regression: Tight Bounds for Stochastic and Online Optimization
• Risk and parameter convergence of logistic regression
• Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes
• First order expansion of convex regularized estimators