Ridge Regularization: An Essential Concept in Data Science

2020/07/06 by Trevor Hastie · 118 citations
Computer Science · Engineering · Mathematics · #Archaeology #Artificial intelligence #Beauty #Computer science #Control Systems and Identification #Craft #Epistemology #Geography #Geology #Image and Signal Denoising Methods #Mathematics #Mathematics education #Paleontology #Philosophy #Regularization (linguistics) #Ridge #Sparse and Compressive Sensing Techniques #Statistics

paper · open access · doi:10.1080/00401706.2020.1791959

published in Technometrics 62(4), 426-433 (Taylor & Francis)

openalex publication_date 2020/07/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/15

Abstract

Ridge or more formally l2 regularization shows up in many areas of statistics and machine learning. It is one of those essential devices that any good data scientist needs to master for their craft. In this brief ridge fest, I have collected together some of the magic and beauty of ridge that my colleagues and I have encountered over the past 40 years in applied statistics.

Citations

• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

Cited by

• Globally aligned Principal Component Analysis for multi-group data
• A Dual-Memory Ferroelectric Transistor Emulating Synaptic Metaplasticity for High-Speed Reservoir Computing
• RAPSEM: Identifying Latent Mediators Without Sequential Ignorability via a Rank-Preserving Structural Equation Model
• Unraveling the neural code: analysis of large-scale two-photon microscopy data
• A Bayesian Framework for Regularized Estimation in Multivariate Models Integrating Approximate Computing Concepts
• A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint
• SLIM-LLMs: Modeling of Style-Sensory Language RelationshipsThrough Low-Dimensional Representations
• Helix 1.0: An Open-Source Framework for Reproducible and Interpretable Machine Learning on Tabular Scientific Data
• Unwrapping photonic reservoirs: enhanced expressivity via random Fourier encoding over stretched domains
• Statistical comparisons of time-series feature sets on classification tasks
• When majority rules, minority loses: bias amplification of gradient descent
• Multimodal domain adaptation under label shift and blockwise missing modalities
• Regularization in Paired Comparison Models via Pseudo-Games and Phantom Players
• Costs for Statutorily Insured Dental Services in Older Germans 2012-2017. [europepmc]
• Anthropogenic influence on extreme precipitation over global land areas seen in multiple observational datasets. [europepmc]
• Predicting Box-Office Markets with Machine Learning Methods. [europepmc]
• The effects of department of Veterans Affairs medical centers on socio-economic outcomes: Evidence from the Paycheck Protection Program. [europepmc]
• Learning from small medical data-robust semi-supervised cancer prognosis classifier with Bayesian variational autoencoder. [europepmc]
• Predictive Model of Liver Toxicity to Aid the Personalized Selection of Proton Versus Photon Therapy in Hepatocellular Carcinoma. [europepmc]
• Comparative Effectiveness Analysis of Lumpectomy and Mastectomy for Elderly Female Breast Cancer Patients: A Deep Learning-based Big Data Analysis. [europepmc]
• A Comparative Investigation of Machine Learning Algorithms for Pore-Influenced Fatigue Life Prediction of Additively Manufactured Inconel 718 Based on a Small Dataset. [europepmc]
• Neurobiologically realistic neural network enables cross-scale modeling of neural dynamics. [europepmc]
• Cross-domain information fusion and personalized recommendation in artificial intelligence recommendation system based on mathematical matrix decomposition. [europepmc]
• Hierarchical joint analysis of marginal summary statistics-Part I: Multipopulation fine mapping and credible set construction. [europepmc]
• Comparison of algorithms using deep reinforcement learning for optimization of hyperbolic metamaterials. [europepmc]
• Unraveling the neural code: analysis of large-scale two-photon microscopy data. [europepmc]
• ARCHI: A New R Package for Automated Imputation of Regionally Correlated Hydrologic Records. [europepmc]
• Real-time 3D temperature reconstruction in microwave cancer hyperthermia from scarce temperature measurements. [europepmc]
• A scoping review on emerging technologies and automation of musculoskeletal ergonomic assessments. [europepmc]
• Stability Analysis of Linear Time Varying Models Constructed from Intracranial EEG Recordings for Modeling Epileptogenic Networks . [europepmc]
• Investigating the temporal dynamics and modeling of mid-level feature representations in humans. [europepmc]
• Helix 1.0: An open-source framework for reproducible and interpretable machine learning on tabular scientific data. [europepmc]
• Predicting time to clearance of sport-related concussions using machine learning. [europepmc]

Related

• Ridge Regularization: an Essential Concept in Data Science
• Lecture notes on ridge regression
• Ridge Regression: Structure, Cross-Validation, and Sketching
• Ridge Regression: Applications to Nonorthogonal Problems
• Ridge regularization for Mean Squared Error Reduction in Regression with Weak Instruments