2024/11/14 by Yantao Wu, Wu, Yantao, Maggioni, Mauro · 1 citation
Chemistry · Engineering · #62G08 #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Spectroscopy and Chemometric Analyses
paper · pdf · doi:10.48550/arxiv.2411.09686
openalex publication_date 2024/11/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Regressing a function F on ℝd without the statistical and computational curse of dimensionality requires special statistical models, for example that impose geometric assumptions on the distribution of the data (e.g., that its support is low-dimensional), or strong smoothness assumptions on F, or a special structure F. Among the latter, compositional models F=f∘ g with g mapping to ℝr with r≪ d include classical single- and multi-index models, as well as neural networks. While the case where g is linear is well-understood, less is known when g is nonlinear, and in particular for which g's the curse of dimensionality in estimating F, or both f and g, may be circumvented. Here we consider a model F(X):=f(ΠγX) where Πγ:ℝd→[0,\textrmlenγ] is the closest-point projection onto the parameter of a regular curve γ:[0, \textrmlenγ]→ℝd, and f:[0,\textrmlenγ]→ ℝ1. The input data X is not low-dimensional: it can be as far from γ as the condition that Πγ(X) is well-defined allows. The distribution X, the curve γ and the function f are all unknown. This model is a natural nonlinear generalization of the single-index model, corresponding to γ being a line. We propose a nonparametric estimator, based on conditional regression, that under suitable assumptions, the strongest of which being that f is coarsely monotone, achieves, up to log factors, the one-dimensional optimal min-max rate for non-parametric regression, up to the level of noise in the observations, and be constructed in time O(d2 nlog n). All the constants in the learning bounds, in the minimal number of samples required for our bounds to hold, and in the computational complexity are at most low-order polynomials in d.