Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
1979/05/01 by Gene H. Golub, Michael Heath, Michael T. Heath +1 · 3,783 citations
Chemistry · Decision Sciences · Mathematics · #Advanced Statistical Methods and Models #Advanced Statistical Process Monitoring #Applied mathematics #Artificial intelligence #Combinatorics #Computer science #Cross-validation #Estimator #Geology #Geometry #Invariant (physics) #Mathematics #Model selection #Regression #Ridge #Rotation (mathematics) #Selection (genetic algorithm) #Spectroscopy and Chemometric Analyses #Statistics #Truncation (statistics) #Value (mathematics)
paper · doi:10.1080/00401706.1979.10489751
published in Technometrics 21(2), 215-223 (Taylor & Francis)
openalex publication_date 1979/05/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
Abstract
Consider the ridge estimate (λ) for β in the model unknown, (λ) = (X T X + nλI)−1 X T y. We study the method of generalized cross-validation (GCV) for choosing a good value for λ from the data. The estimate is the minimizer of V(λ) given by where A(λ) = X(X T X + nλI)−1 X T . This estimate is a rotation-invariant version of Allen's PRESS, or ordinary cross-validation. This estimate behaves like a risk improvement estimator, but does not require an estimate of σ2, so can be used when n − p is small, or even if p ≥ 2 n in certain cases. The GCV method can also be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.
Citations
Cited by
- A Statistical View of Some Chemometrics Regression Tools
- Ridge Regularization: An Essential Concept in Data Science
- Better Subset Regression Using the Nonnegative Garrote
- Multi-objective optimization-inspired set theory-based regularization approach for force reconstruction with bounded uncertainties
- Modelling non‐linear psychological processes: Reviewing and evaluating non‐parametric approaches and their applicability to intensive longitudinal data
- Normalized signal-to-noise ratio for covariance estimation in target detection
- Learning Lévy density via adaptive RKHS regression with bi-level optimization
- Approximate cross-validation formula for Bayesian linear regression
- Localizing differences in smooths with simultaneous confidence bounds on\n the true discovery proportion
- SimPEG: An open source framework for simulation and gradient based parameter estimation in geophysical applications
- Simultaneous Identification of Coefficient and Initial State for One-Dimensional Heat Equation from Boundary Control and Measurement
- Learning Regularization Structure for Biosignal Template Estimation
- Unbiased Bregman-Risk Estimators: Application to Regularization Parameter Selection in Tomographic Image Reconstruction
- Generative Bayesian Hyperparameter Tuning
- Ridge Estimation-Based Vision and Laser Ranging Fusion Localization Method for UAVs
- Convex Techniques for Model Selection
- Deep Learning in Neural Networks: An Overview
- The Degrees of Freedom of the Group Lasso
- Synthesis imaging with a lunar orbit array: I. global sky map and its systematics
- Learning stochasticity: a nonparametric framework for intrinsic noise estimation
- Flat Minima
- Numerically Efficient and Stable Algorithms for Kernel-Based Regularized System Identification Using Givens-Vector Representation
- Efficient Krylov-Regularization Solvers for Multiquadric RBF Discretizations of the 3D Helmholtz Equation
- Comparing EPGP Surrogates and Finite Elements Under Degree-of-Freedom Parity
- Manifold-regression to predict from MEG/EEG brain signals without source modeling
- Risk Based Arsenic Rational Sampling Design for Public and Environmental Health Management
- Longitudinal Dynamic Functional Regression
- Greedy metrics in orthogonal greedy learning
- Projective Graph Residualization: Variation-Allocation Frontiers for Control-Function IV
- Detecting Environment-Dependent Diversification From Phylogenies: A Simulation Study and Some Empirical Illustrations
- Generalized Kalman Smoothing: Modeling and Algorithms
- Improvement of code behaviour in a design of experiments by metamodeling
- Computational methods for large-scale inverse problems: a survey on hybrid projection methods
- Tomographic absorption spectroscopy for the study of gas dynamics and reactive flows
- A stochastic extended Rippa's algorithm for LpOCV
- Functional Accelerated Failure Time Models for Predicting Time Since Cannabis Use
- Regularization and the small-ball method II: complexity dependent error rates
- Generalized Additive Models
- Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learning
- Hybrid Projection Methods for Large-scale Inverse Problems with Mixed Gaussian Priors
- Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics
- Optimal smoothing parameter in Eilers-Whittaker smoother
- Maximum likelihood estimation of regularisation parameters in high-dimensional inverse problems: an empirical Bayesian approach. Part I: Methodology and Experiments
- Consistent Risk Estimation in Moderately High-Dimensional Linear Regression
- Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation
- Gated X-TFC: Soft Domain Decomposition for Forward and Inverse Problems in Sharp-Gradient PDEs
- Reproducibility of fNIRS within subject for visual and motor tasks
- Adjusted chi-square test for degree-corrected block models
- Generalized Orthogonal Components Regression for High Dimensional Generalized Linear Models
- The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems
- Avoiding The Double Descent Phenomenon of Random Feature Models Using Hybrid Regularization
- Just Interpolate: Kernel "Ridgeless" Regression Can Generalize
- Fast transforms for high order boundary conditions
- AHP-Net: adaptive-hyper-parameter deep learning based image reconstruction method for multilevel low-dose CT
- Discrete cosine transform LSQR and GMRES methods for multidimensional ill-posed problems
- Convergence and Generalization of Anti-Regularization for Parametric Models
- A Parameter Choice Strategy for the Inversion of Multiple Observations
- A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning
- Non-uniform refinement: adaptive regularization improves single-particle cryo-EM reconstruction
- The Regularization Theory of the Krylov Iterative Solvers LSQR, CGLS, LSMR and CGME For Linear Discrete Ill-Posed Problems
- Time-domain sound field estimation using kernel ridge regression
- Reconstruction of the Dipole Amplitude in the Dipole Picture as a mathematical Inverse Problem
- HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognition
- Fused Spatial Point Process Intensity Estimation with Varying Coefficients on Complex Constrained Domains
- Supplementary Material for CDC Submission No. 1461
- Q-curve and area rules for choosing heuristic parameter in Tikhonov regularization
- Empirical Risk Minimization as Parameter Choice Rule for General Linear\n Regularization Methods
- Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion
- A semi-automatic method to guide the choice of ridge parameter in ridge regression
- Constructive algorithms for structure learning in feedforward neural networks for regression problems
- Efficient Numerical Optimization For Susceptibility Artifact Correction Of EPI-MRI
- Uncertainty-informed regionalization of sub-daily rainfall extremes in northwestern Italy using Bilinear Surface Smoothing with credible-interval-based data screening
- A random model for multidimensional fitting method
- High-dimensional Gaussian graphical model for network-linked data
- An automatic procedure to determine groups of nonparametric regression curves
- A Levinson-Galerkin algorithm for regularized trigonometric approximation
- Advances and challenges of the Conditional Source-term Estimation model for turbulent reacting flows
- Functional Horseshoe Priors for Subspace Shrinkage
- A scalable estimate of the extra-sample prediction error via approximate leave-one-out
- Régularisation dans les Modèles Linéaires Généralisés Mixtes avec effet aléatoire autorégressif
- Whiteout: when do fixed-X knockoffs fail?
- Linear screening for high-dimensional computer experiments
- Automatic reproducing kernel and regularization for learning convolution kernels
- Estimation of Kullback-Leibler losses for noisy recovery problems within the exponential family
- Fast inference in generalized linear models via expected log-likelihoods
- A Non-Convex Optimization Technique for Sparse Blind Deconvolution -- Initialization Aspects and Error Reduction Properties
- Fast Cross-Validation for Incremental Learning
- A Two-Stage Penalized Least Squares Method for Constructing Large Systems of Structural Equations
- ROS Regression: Integrating Regularization and Optimal Scaling Regression
- Distribution-dependent Generalization Bounds for Tuning Linear Regression Across Tasks
- On the Predictive Risk in Misspecified Quantile Regression
- Comparisons of penalized least squares methods by simulations
- Quantum Regularized Least Squares Solver with Parameter Estimate
- Optimal Regularization Parameters for General-Form Tikhonov Regularization
- Tensor GMRES and Golub-Kahan Bidiagonalization methods via the Einstein product with applications to image and video processing
- Greedy Criterion in Orthogonal Greedy Learning
- The cost-free nature of optimally tuning Tikhonov regularizers and other ordered smoothers
- GCV for Tikhonov regularization by partial SVD
- A data-driven convergence criterion for iterative unfolding of smeared spectra
- The maximum penalty criterion for ridge regression: application to the calibration of the force constant in elastic network models
- Optimal hybrid parameter selection for stable sequential solution of inverse heat conduction problem
- Cross-validation of matching correlation analysis by resampling matching weights
- Kernel Regularized Least Squares: Reducing Misspecification Bias with a Flexible and Interpretable Machine Learning Approach
- TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems
- A parallel sampling algorithm for inverse problems with linear and nonlinear unknowns
- A New Version of a Posteriori Choosing Regularization Parameter in\n Ill-Posed Problems
- The discrete picard condition for discrete ill-posed problems
- CVEK: Robust Estimation and Testing for Nonlinear Effects using Kernel Machine Ensemble
- Bayesian Field Theory: Nonparametric Approaches to Density Estimation, Regression, Classification, and Inverse Quantum Problems
- On Optimal Generalizability in Parametric Learning
- Data-driven research on chemical features of Jingdezhen and Longquan celadon by energy dispersive X-ray fluorescence
- Solving Implicit Inverse Problems with Homotopy-Based Regularization Path
- Decision Theoretic Bootstrapping
- Cross Validation for Penalized Quantile Regression with a Case-Weight Adjusted Solution Path
- Factor-augmented Smoothing Model for Functional Data
- Robust smoothing of gridded data in one and higher dimensions with missing values
- The regularization theory of the Krylov iterative solvers LSQR and CGLS for linear discrete ill-posed problems, part I: the simple singular value case
- Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems
- Transformer learns the cross-task prior and regularization for in-context learning
- Numerical reconstructions of a source term in a mobile-immobile diffusion model from the partial interior observation
- Training NTK to Generalize with KARE
- Is Supervised Learning Really That Different from Unsupervised?
- A GCV based Arnoldi-Tikhonov regularization method
- Reduced rank regression via adaptive nuclear norm penalization
- Learning Neural Networks with Adaptive Regularization
- From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation
- A Bayesian Algorithm for Reconstructing Climate Anomalies in Space and Time. Part II: Comparison with the Regularized Expectation–Maximization Algorithm
- Fast Multitask Gaussian Process Regression
- Local time resolved dynamics of field‐aligned currents and their response to solar wind variability
- Fast Automatic Bayesian Cubature Using Lattice Sampling
- On the emergence of numerical instabilities in Next Generation Reservoir Computing
- Fast Compute for ML Optimization
- Newton-Puiseux Analysis for Interpretability and Calibration of Complex-Valued Neural Networks
- The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses
- Orthogonal reparametrization of the Nelson-Siegel-Svensson interest rate curve model: conditioning, diagnostics, and identifiability
- A Time-Evolving 3D Method Dedicated to the Reconstruction of Solar Plumes and Results Using Extreme Ultraviolet Data
- Model Selection Techniques -- An Overview
- Multivariate convex regression with adaptive partitioning
- Model Error Covariance Estimation for Weak Constraint Data Assimilation
- Linear shrinkage estimation of covariance matrices using low-complexity cross-validation
- Efficient Bandwidth Estimation in 2D Filtered Backprojection Reconstruction
- Regularization and Selection in A Directed Network Model with Nodal Homophily and Nodal Effects
- A Bidiagonalization-Regularization Procedure for Large Scale Discretizations of Ill-Posed Problems
- Efficient Testing Using Surrogate Information
- Information-Corrected Estimation: A Generalization Error Reducing Parameter Estimation Method
- First antineutrino energy spectrum from 235 U fissions with the S TEREO detector at ILL *
- Unsupervised frequency tracking beyond the Nyquist frequency using Markov chains
- Regularized adaptive long autoregressive spectral analysis
- Combining predictions from linear models when training and test inputs\n differ
- Network structure effects in reservoir computers
- A recipe for accurate estimation of lifespan brain trajectories, distinguishing longitudinal and cohort effects
- Adaptive Bayesian Radio Tomography
- The unified maximum a posteriori (MAP) framework for neuronal system identification
- Ridge regression [wikipedia]
- Robust and efficient parameter estimation in dynamic models of biological systems. [europepmc]
- Combining energy and Laplacian regularization to accurately retrieve the depth of brain activity of diffuse optical tomographic data. [europepmc]
- Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis. [europepmc]
- PyLDM - An open source package for lifetime density analysis of time-resolved spectroscopic data. [europepmc]
- Mechanical interactions among followers determine the emergence of leaders in migrating epithelial cell collectives. [europepmc]
- Evaluation of Fifteen Algorithms for the Resolution of the Electrocardiography Imaging Inverse Problem Using ex-vivo and in-silico Data. [europepmc]
- Comparison of source localization techniques in diffuse optical tomography for fNIRS application using a realistic head model. [europepmc]
- Traction force microscopy with optimized regularization and automated Bayesian parameter selection for comparing cells. [europepmc]
- A review of spline function procedures in R. [europepmc]
- Skin Conductance as a Viable Alternative for Closing the Deep Brain Stimulation Loop in Neuropsychiatric Disorders. [europepmc]
- Magnetic resonance measurements of cellular and sub-cellular membrane structures in live and fixed neural tissue. [europepmc]
- Connecting concepts in the brain by mapping cortical representations of semantic relations. [europepmc]
- Combining magnetoencephalography with magnetic resonance imaging enhances learning of surrogate-biomarkers. [europepmc]
- Temporal selectivity declines in the aging human auditory cortex. [europepmc]
- Ridge regression and its applications in genetic studies. [europepmc]
- LASSO type penalized spline regression for binary data. [europepmc]
- Continuous diffusion spectrum computation for diffusion-weighted magnetic resonance imaging of the kidney tubule system. [europepmc]
- DeerLab: a comprehensive software package for analyzing dipolar electron paramagnetic resonance spectroscopy data. [europepmc]
- To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. [europepmc]
- Wearable Sensor-Based Prediction Model of Timed up and Go Test in Older Adults. [europepmc]
- Comparison of direct and inverse methods for 2.5D traction force microscopy. [europepmc]
- Breaking the resolution-bandwidth limit of chip-scale spectrometry by harnessing a dispersion-engineered photonic molecule. [europepmc]
- Complex modeling with detailed temporal predictors does not improve health records-based suicide risk prediction. [europepmc]
- Ultra-simplified diffraction-based computational spectrometer. [europepmc]
- Dendritic excitations govern back-propagation via a spike-rate accelerometer. [europepmc]
Related