The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction

1974/02/01 by David M. Allen · 1,388 citations
Decision Sciences · Engineering · Mathematics · #Advanced Statistical Methods and Models #Advanced Statistical Process Monitoring #Artificial intelligence #Computer science #Data mining #Econometrics #Engineering #Feature selection #Limiting #Linear regression #Mathematics #Regression #Regression analysis #Ridge #Selection (genetic algorithm) #Statistical Methods and Inference #Statistics #Variable (mathematics)

paper · doi:10.1080/00401706.1974.10489157

published in Technometrics 16(1), 125-127 (Taylor & Francis)

openalex publication_date 1974/02/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We show that data augmentation provides a rather general formulation for the study of biased prediction techniques using multiple linear regression. Variable selection is a limiting case, and Ridge regression is a special case of data augmentation. We propose a way to obtain predictors given a credible criterion of good prediction.

Citations

• Ridge Regression: Applications to Nonorthogonal Problems

Cited by

• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
• Simultaneous Optimization of Multiple Responses Represented by Polynomial Regression Functions
• Machine Learning for Electrode Materials: Property Prediction via Composition
• Approximate cross-validation formula for Bayesian linear regression
• Optimizing Approximate Leave-one-out Cross-validation to Tune Hyperparameters
• On the Accuracy of Newton Step and Influence Function Data Attributions
• True Few-Shot Learning with Language Models
• Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression
• Cross-Validation: What Does It Estimate and How Well Does It Do It?
• Estimating Subagging by cross-validation
• Contributors of carbon dioxide in the atmosphere in Europe
• Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
• Veridical Data Science
• Concentration inequalities of the cross-validation estimate for stable predictors
• Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
• Model Selection by Loss Rank for Classification and Unsupervised Learning
• Consistent Risk Estimation in Moderately High-Dimensional Linear Regression
• V-fold cross-validation improved: V-fold penalization
• Pharmacokinetic Equations Applied to Obtain New Topological Models in the Search of Antibacterial Compounds
• Model selection for ecologists: the worldviews of AIC and BIC
• Prediction Measures in Nonlinear Beta Regression Models
• A novel hybrid trust region algorithm based on nonmonotone and LOOCV techniques
• Estimating the genetic architecture of quantitative traits
• Prediction Measures in Beta Regression Models
• Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence
• Irredundant k-Fold Cross-Validation
• Comparing Classes of Estimators: When does Gradient Descent Beat Ridge Regression in Linear Models?
• A random model for multidimensional fitting method
• A scalable estimate of the extra-sample prediction error via approximate leave-one-out
• Data-driven calibration of linear estimators with minimal penalties
• Systematic QSAR Procedures with Quantum Chemical Descriptors
• The optimal Padé polynomial for reconstruction of luminosity distance based on 10-fold cross-validation
• Cross-Validation with Confidence
• Marginalizing Corrupted Features
• Choosing a penalty for model selection in heteroscedastic regression
• ASCII: ASsisted Classification with Ignorance Interchange
• Approximate Leave-One-Out for Fast Parameter Tuning in High Dimensions
• Survival prediction of head and neck squamous cell carcinoma using machine learning models
• Robust Locally Weighted Regression and Smoothing Scatterplots
• On Variable Screening in Multiple Nonparametric Regression Model
• On Optimal Generalizability in Parametric Learning
• PANDA: AdaPtive Noisy Data Augmentation for Regularization of Undirected Graphical Models
• LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression
• Decision Theoretic Bootstrapping
• The vacuolar sulfate transporter PsSULTR4 is a key determinant of seed yield and protein composition in pea
• Is Supervised Learning Really That Different from Unsupervised?
• Choice of V for V-Fold Cross-Validation in Least-Squares Density Estimation
• The Risk Inflation Criterion for Multiple Regression
• An Evaluation of the Hybrid Model for Predicting Surgery Duration
• Joint Model and Data Sparsification via the Marginal Likelihood
• The Akaike information criterion: Background, derivation, properties, application, interpretation, and refinements
• Model Selection Techniques -- An Overview
• Regression and time series model selection in small samples
• Application of quantum machine learning using the quantum kernel algorithm on high energy physics analysis at the LHC
• Deep neural network for solving differential equations motivated by Legendre-Galerkin approximation
• Measurement of CP Violation in the Decay B+→K+π0
• A Variational View on Statistical Multiscale Estimation
• Consistency of cross validation for comparing regression procedures
• A Better Alternative to Piecewise Linear Time Series Segmentation
• The Omnicausal Model Reveals the Highly Polyfactorial Nature of Complex Diseases
• Predicting Defect Content and Quality Assurance Effectiveness by Combining Expert Judgment and Defect Data - A Case Study
• Cross-validation pitfalls when selecting and assessing regression and classification models. [europepmc]
• Efficient strategies for leave-one-out cross validation for genomic best linear unbiased prediction. [europepmc]
• Predictive models of poly(ethylene-terephthalate) film degradation under multi-factor accelerated weathering exposures. [europepmc]
• Antiviral Properties of Chemical Inhibitors of Cellular Anti-Apoptotic Bcl-2 Proteins. [europepmc]
• A Modified Tri-Exponential Model for Multi- b -value Diffusion-Weighted Imaging: A Method to Detect the Strictly Diffusion-Limited Compartment in Brain. [europepmc]
• Developing an Accurate and Fast Non-Destructive Single Leaf Area Model for Loquat ( Eriobotrya japonica Lindl) Cultivars. [europepmc]
• Predicting Dengue Outbreaks in Cambodia. [europepmc]
• Accessory pathway analysis using a multimodal deep learning model. [europepmc]
• Connecting Social Psychology and Deep Reinforcement Learning: A Probabilistic Predictor on the Intention to Do Home-Based Physical Activity After Message Exposure. [europepmc]
• Collective variable-based enhanced sampling and machine learning. [europepmc]
• Gelatin-Alginate Coacervates Optimized by DOE to Improve Delivery of bFGF for Wound Healing. [europepmc]
• Socio-cultural similarity with host population rather than ecological similarity predicts success and failure of human migrations. [europepmc]
• Barcoded bulk QTL mapping reveals highly polygenic and epistatic architecture of complex traits in yeast. [europepmc]
• Dysregulation of Human Somatic piRNA Expression in Parkinson's Disease Subtypes and Stages. [europepmc]
• Adult Skeletal Age-at-Death Estimation through Deep Random Neural Networks: A New Method and Its Computational Analysis. [europepmc]
• Weekly Nowcasting of New COVID-19 Cases Using Past Viral Load Measurements. [europepmc]
• Clinical studies of detecting COVID-19 from exhaled breath with electronic nose. [europepmc]
• Structural gray matter features and behavioral preliterate skills predict future literacy - A machine learning approach. [europepmc]
• Quantifying portable genetic effects and improving cross-ancestry genetic prediction with GWAS summary statistics. [europepmc]
• Raman spectroscopy and topological machine learning for cancer grading. [europepmc]
• County-Level Prevalence Estimates of Autism Spectrum Disorder in Children in the United States. [europepmc]
• FLUID-GPT (Fast Learning to Understand and Investigate Dynamics with a Generative Pre-Trained Transformer): Efficient Predictions of Particle Trajectories and Erosion. [europepmc]
• A text mining approach to categorize patient safety event reports by medication error type. [europepmc]
• Clinical Characteristics and Outcomes of Drug-Induced Acute Kidney Injury Cases. [europepmc]
• Study of PARP inhibitors for breast cancer based on enhanced multiple kernel function SVR with PSO. [europepmc]

Related

• A note on selection stability: combining stability and prediction
• Variable Selection in Seemingly Unrelated Regressions with Random Predictors
• Estimating the theoretical error rate for prediction
• Variable Selection in High Dimensional Linear Regressions with Parameter Instability
• Optimal Two-Step Prediction in Regression