2025/10/25 by K. E. Chang, Chang, Kaihua
Computer Science · #Stochastic Gradient Optimization Techniques #Adversarial Robustness in Machine Learning #Gaussian Processes and Bayesian Inference
paper · pdf · doi:10.48550/arxiv.2510.22419
Quantile regression is fundamental to distributional modeling, yet independent estimation of multiple quantiles frequently produces crossing -- where estimated quantile functions violate monotonicity, implying impossible negative probability densities. While Constrained Joint Quantile Regression (CJQR) elegantly enforces non-crossing by construction, existing formulations via Linear Programming exhibit O((qn)3) complexity, rendering them impractical for large-scale applications. We present the first scalable solution using PyTorch automatic differentiation: CJQR-ALM, combining the Augmented Lagrangian Method with differentiable pinball loss and L-BFGS optimization. Our approach reduces computational complexity to O(n), achieving near-zero crossing rates on datasets exceeding 70,000 observations within minutes. The differentiable formulation naturally extends to neural network architectures for non-linear conditional quantile estimation. Application to Student Growth Percentile calculations demonstrates practical utility for educational assessment, while simulation studies show negligible accuracy cost (RMSE increase ≈ 2.4 points) relative to unconstrained estimation -- a favorable trade-off for applications requiring valid probability statements across finance, healthcare, and engineering.