Consistent Nonparametric Regression

1977/07/01 by Charles J. Stone · 31 citations
Computer Science · Mathematics · #Advanced Statistical Methods and Models #Bayesian Modeling and Causal Inference #Statistical Methods and Inference

paper · pdf · doi:10.1214/aos/1176343886

openalex publication_date 1977/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

Let (X, Y) be a pair of random variables such that X is ℝ^d-valued and Y is ℝ^d'-valued. Given a random sample (X₁, Y₁), ⋯, (X_n, Y_n) from the distribution of (X, Y), the conditional distribution P^Y(\bullet | X) of Y given X can be estimated nonparametrically by P_n^Y(A | X) = ∑ⁿ₁ W_ni(X)I_A(Y_i), where the weight function W_n is of the form W_ni(X) = W_ni(X, X₁, ⋯, X_n), 1 \leqq i \leqq n. The weight function W_n is called a probability weight function if it is nonnegative and ∑ⁿ₁ W_ni(X) = 1. Associated with P_n^Y(\bullet | X) in a natural way are nonparametric estimators of conditional expectations, variances, covariances, standard deviations, correlations and quantiles and nonparametric approximate Bayes rules in prediction and multiple classification problems. Consistency of a sequence \W_n\ of weight functions is defined and sufficient conditions for consistency are obtained. When applied to sequences of probability weight functions, these conditions are both necessary and sufficient. Consistent sequences of probability weight functions defined in terms of nearest neighbors are constructed. The results are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.

Citations

• Remarks on Some Nonparametric Estimates of a Density Function
• The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data
• Nearest neighbor pattern classification

Cited by

• PrivAR: Client-Side Privacy Framework for Real-Time Location-Based Augmented Reality
• Local polynomial estimation of quantile density functions
• Bellman Calibration for V-Learning in Offline Reinforcement Learning
• Universal consistency of the k-NN rule in metric spaces and Nagata dimension. III
• Frequentist Persuasion
• End-to-End Test-Time Training for Long Context
• Optimal Sampling Density for Nonparametric Regression
• Improving the prediction accuracy of statistical models: A new hierarchical clustering approach
• Nonparametric bootstrap for propensity score matching estimators
• Monte Carlo on a single sample
• A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds
• Some Robustness Properties of Label Cleaning
• kNNSampler: Stochastic Imputations for Recovering Missing Value Distributions
• Hierarchical Mixtures of Experts and the EM Algorithm
• Closed Form Estimators for a Class of Semiparametric Multinomial Choice Models
• Sufficient Representations for Categorical Variables
• Not All are Made Equal: Consistency of Weighted Averaging Estimators Under Active Learning
• Cost-Aware Contrastive Routing for LLMs
• Adaptive Non-Parametric Regression With the K-NN Fused Lasso
• Nonparametric Estimation For Censored Circular Data
• Expressivity of expand-and-sparsify representations
• Distributed Adaptive Nearest Neighbor Classifier: Algorithm and Theory
• Stone's theorem for distributional regression in Wasserstein distance
• Predictive Interval Models for Non-parametric Regression
• Shuffled Graph Classification: Theory and Connectome Applications
• Discriminative Learning of Similarity and Group Equivariant\n Representations
• Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes
• Local Linear Forests
• A partition-based similarity for classification distributions
• Classification with the nearest neighbor rule in general finite dimensional spaces: necessary and sufficient conditions
• Conformal prediction intervals for the individual treatment effect

Related

• Confidence intervals for nonparametric regression
• Uniform Consistency in Nonparametric Mixture Models
• A Theoretical Framework for Bayesian Nonparametric Regression
• Nonparametric Regression, Confidence Regions and Regularization
• A Nonparametric Maximum Likelihood Approach to Mixture of Regression