A Statistical View of Some Chemometrics Regression Tools

1993/05/01 by lldiko E. Frank, Jerome H. Friedman · 2,238 citations
Chemistry · Computer Science · Mathematics · #Advanced Statistical Methods and Models #Chemometrics #Computational Drug Discovery Methods #Computer science #Data mining #Econometrics #Field (mathematics) #Machine learning #Mathematics #Ordinary least squares #Partial least squares regression #Principal component analysis #Principal component regression #Regression analysis #Spectroscopy and Chemometric Analyses #Statistics

paper · doi:10.1080/00401706.1993.10485033

published in Technometrics 35(2), 109-135 (Taylor & Francis)

openalex publication_date 1993/05/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis. In addition to borrowing many techniques from the statistics and engineering literatures, chemometrics itself has given rise to several new data-analytical methods. This article examines two methods commonly used in chemometrics for predictive modeling—partial least squares and principal components regression—from a statistical perspective. The goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations. These methods include ordinary least squares, variable subset selection, and ridge regression.

Citations

• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
• Smoothing noisy data with spline functions

Cited by

• Better Subset Regression Using the Nonnegative Garrote
• The composite absolute penalties family for grouped and hierarchical variable selection
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• CANN-EUCLID: unsupervised constitutive artificial neural network model discovery from full-field data
• Complexity of Unconstrained L₂-L_p Minimization
• Enhancing precision in human neuroscience
• Multinomial Inverse Regression for Text Analysis
• Generalized Hoeffding-Sobol Decomposition for Dependent Variables -Application to Sensitivity Analysis
• Principal Component Projection Without Principal Component Analysis
• Bayesian Global-Local Regularization
• Predicting key soil properties from Vis-NIR spectra by applying dual-wavelength indices transformations and stacking machine learning approaches
• Calibration and partial calibration on principal components when the number of auxiliary variables is large
• High-Dimensional Methods and Inference on Structural and Treatment Effects
• Bayesian Bridge Gaussian Process Regression
• Transformed ℓ₁ Gradient Regularization for Image Denoising
• Kernel partial least squares for stationary data
• Strong NP-Hardness for Sparse Optimization with Concave Penalty Functions
• Tensor Regression with Applications in Neuroimaging Data Analysis
• Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators
• On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothness
• Sequential Convex Programming Methods for A Class of Structured Nonlinear Programming
• Stochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization
• The Degrees of Freedom of Partial Least Squares Regression
• The geometry of PLS shrinkages
• Selective Factor Extraction in High Dimensions
• The Perceived Influences of Environment on Health in Italy: a Penalized Ordinal Regression Approach
• Comparison of Gaussian process regression, partial least squares, random forest and support vector machines for a near infrared calibration of paracetamol samples
• An algorithmic view of ℓ₂ regularization and some path-following algorithms
• Closed-form ℓ_r norm scaling with data for overparameterized linear regression and diagonal linear networks under ℓ_p bias
• Penalized regression with correlation-based penalty
• Modelling extensions for multi-location studies in environmental epidemiology
• The GMO90+ Project: Absence of Evidence for Biologically Meaningful Effects of Genetically Modified Maize-based Diets on Wistar Rats After 6-Months Feeding Comparative Trial
• Bayesian L_(1)/(2) regression
• On the Consistency of the Bias Correction Term of the AIC for the Non-Concave Penalized Likelihood Method
• Matrix Completion with Nonconvex Regularization: Spectral Operators and Scalable Algorithms
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• Nonconcave penalized likelihood with a diverging number of parameters
• Biomarkers selection and combination based on the weighted Youden index
• Penalized Regressions: The Bridge versus the Lasso
• Rank-one Convexification for Sparse Regression
• A unified framework for the study of the PLS estimator's properties
• Low noise sensitivity analysis of Lq-minimization in oversampled systems
• Material Fingerprinting: A shortcut to material model discovery without solving optimization problems
• Strategic Bayesian Asset Allocation
• Penalized robust estimators in logistic regression with applications to sparse models
• Generalized Thresholding and Online Sparsity-Aware Learning in a Union of Subspaces
• Penalized Likelihood Estimation in High-Dimensional Time Series Models and its Application
• A semi-automatic method to guide the choice of ridge parameter in ridge regression
• Variable Selection for Survival Data with A Class of Adaptive Elastic Net Techniques
• Partial least squares for dependent data
• Deep Learning Partial Least Squares
• On rate optimal local estimation in functional linear model
• Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysis
• Tensor Generalized Estimating Equations for Longitudinal Imaging Analysis
• On optimality of Bayesian testimation in the normal means problem
• Neural Text Embeddings in Psychological Research: A Guide With Examples in R
• Path Following and Empirical Bayes Model Selection for Sparse Regression
• Functional linear regression with derivatives
• Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes
• A Two-Stage Penalized Least Squares Method for Constructing Large Systems of Structural Equations
• Algorithmic Versatility of SPF-regularization Methods
• An Alternative Approach to Functional Linear Partial Quantile Regression
• ROS Regression: Integrating Regularization and Optimal Scaling Regression
• Non-smooth optimization meets automated material model discovery
• The Effects of Flipped Classrooms in Higher Education: A Causal Machine Learning Analysis
• Penalty, Shrinkage, and Preliminary Test Estimators under Full Model Hypothesis
• On estimating covariances between many assets with histories of highly variable length
• Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
• Overcoming The Limitations of Phase Transition by Higher Order Analysis of Regularization Techniques
• The Variational Garrote
• Scientific Applications Leveraging Randomized Linear Algebra
• A Functional Hodrick Prescott Filter
• Local Linear Regression on Manifolds and its Geometric Interpretation
• A Nonconvex Nonsmooth Regularization Method for Compressed Sensing and Low-Rank Matrix Completion
• Tree-based boosting with functional data
• EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning
• An Exploratory Analysis of Biased Learners in Soft-Sensing Frames
• Robust Variable Selection in High-dimensional Nonparametric Additive Model
• A consistent estimator of the smoothing operator in the functional Hodrick-Prescott filter
• Multi-Tensor Network Representation for High-Order Tensor Completion
• Global Minimizers of ℓ^p-Regularized Objectives Yield the Sparsest ReLU Neural Networks
• Deterministic Stretchy Regression
• From deep to Shallow: Equivalent Forms of Deep Networks in Reproducing Kernel Krein Space and Indefinite Support Vector Machines
• Asymptotic distribution and sparsistency for l1-penalized parametric M-estimators with applications to linear SVM and logistic regression
• The Trimmed Lasso: Sparsity and Robustness
• Low-Rank and Total Variation Regularization and Its Application to Image Recovery
• De-noising procedures for frame operators
• A Selective Review of Group Selection in High-Dimensional Models
• Least angle and ℓ1 penalized regression: A review
• Comparing two samples by penalized logistic regression
• Sparse estimators and the oracle property, or the return of Hodges’ estimator
• Variable selection in semiparametric regression modeling
• One-step sparse estimates in nonconcave penalized likelihood models
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Efficient algorithm to select tuning parameters in sparse regression modeling with regularization
• Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
• Robust rank correlation based screening
• Bayesian shrinkage methods for partially observed data with many predictors
• Data-driven Soft Sensors in the process industry
• Leveraging loyalty programs using competitor based targeting
• Multivariate calibration with temperature interaction using two-dimensional penalized signal regression
• PLS-regression: a basic tool of chemometrics
• Prediction by Supervised Principal Components
• Shrinkage Structure of Partial Least Squares
• Adaptive Sparse Group Lasso Penalized Quantile Regression via Dual ADMM
• Statistical methods for body mass index: a selective review of the literature
• Reduced Rank Multivariate Kernel Ridge Regression
• The Problem of Overfitting
• Variable selection in nonparametric additive models
• Learning Effective Loss Functions Efficiently
• On Hodges’ superefficiency and merits of oracle property in model selection
• Selection of tuning parameters in bridge regression models via Bayesian information criterion
• Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis
• Fully Bayesian Penalized Regression with a Generalized Bridge Prior
• On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Unsupervised discovery of interpretable hyperelastic constitutive laws
• Partial least squares regression [wikipedia]
• Principal component regression [wikipedia]
• Predictive algorithms for determination of reflectance data from quantity of pigments within experimental dental resin composites. [europepmc]
• Rare variants analysis using penalization methods for whole genome sequence data. [europepmc]
• EM Adaptive LASSO-A Multilocus Modeling Strategy for Detecting SNPs Associated with Zero-inflated Count Phenotypes. [europepmc]
• Bootstrap Enhanced Penalized Regression for Variable Selection with Neuroimaging Data. [europepmc]
• Evaluation of Classifier Performance for Multiclass Phenotype Discrimination in Untargeted Metabolomics. [europepmc]
• Pattern Discovery in Brain Imaging Genetics via SCCA Modeling with a Generic Non-convex Penalty. [europepmc]
• Time-Gated Raman Spectroscopy for Quantitative Determination of Solid-State Forms of Fluorescent Pharmaceuticals. [europepmc]
• Extended Near-Infrared Optoacoustic Spectrometry for Sensing Physiological Concentrations of Glucose. [europepmc]
• Genomic selection using principal component regression. [europepmc]
• Temperature and soil moisture control microbial community composition in an arctic-alpine ecosystem along elevational and micro-topographic gradients. [europepmc]
• Combining multiple connectomes improves predictive modeling of phenotypic measures. [europepmc]
• Multitable Methods for Microbiome Data Integration. [europepmc]
• CoMPARA: Collaborative Modeling Project for Androgen Receptor Activity. [europepmc]
• Exploratory study on classification of lung cancer subtypes through a combined K-nearest neighbor classifier in breathomics. [europepmc]
• The Sample, the Spectra and the Maths-The Critical Pillars in the Development of Robust and Sound Applications of Vibrational Spectroscopy. [europepmc]
• Shallow neural networks for fluid flow reconstruction with limited sensors. [europepmc]
• Risk Prediction in Patients With Heart Failure With Preserved Ejection Fraction Using Gene Expression Data and Machine Learning. [europepmc]
• CATMoS: Collaborative Acute Toxicity Modeling Suite. [europepmc]
• Asymmetric effects of economic policy uncertainty and environmental policy stringency on environmental quality: evidence from China and the United States. [europepmc]
• Traits, phylogeny and host cell receptors predict Ebolavirus host status among African mammals. [europepmc]
• Enhancing precision in human neuroscience. [europepmc]
• Genomic prediction using machine learning: a comparison of the performance of regularized regression, ensemble, instance-based and deep learning methods on synthetic and empirical data. [europepmc]
• Comparative effectiveness of second line oral antidiabetic treatments among people with type 2 diabetes mellitus: emulation of a target trial using routinely collected health data. [europepmc]
• Left Atrial Roof Enlargement Is a Distinct Feature of Heart Failure With Preserved Ejection Fraction. [europepmc]
• Democratizing biomedical simulation through automated model discovery and a universal material subroutine. [europepmc]

Related

• Dual-sPLS: A family of Dual Sparse Partial Least Squares regressions for feature selection and prediction with tunable sparsity; evaluation on simulated and near-infrared (NIR) data
• Utilization of Partial Least Squares for Simultaneous Feature Selection and Extraction
• Variable selection with genetic algorithms using repeated cross-validation of PLS regression models as fitness measure
• Complex-Valued Chemometrics in Spectroscopy: Classical Least Squares Regression
• When are novel methods for analyzing complex chemical mixtures in epidemiology beneficial?