End-to-End Probabilistic Framework for Learning with Hard Constraints

2025/06/08 by Utkarsh Utkarsh, Utkarsh, Utkarsh, Danielle C. Maddix +7
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Gaussian Processes and Bayesian Inference #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.2506.07003

Abstract

We present ProbHardE2E, a probabilistic forecasting framework that incorporates hard operational/physical constraints, and provides uncertainty quantification. Our methodology uses a novel differentiable probabilistic projection layer (DPPL) that can be combined with a wide range of neural network architectures. DPPL allows the model to learn the system in an end-to-end manner, compared to other approaches where constraints are satisfied either through a post-processing step or at inference. ProbHardE2E optimizes a strictly proper scoring rule, without making any distributional assumptions on the target, which enables it to obtain robust distributional estimates (in contrast to existing approaches that generally optimize likelihood-based objectives, which are heavily biased by their distributional assumptions and model choices); and it can incorporate a range of non-linear constraints (increasing the power of modeling and flexibility). We apply ProbHardE2E in learning partial differential equations with uncertainty estimates and to probabilistic time-series forecasting, showcasing it as a broadly applicable general framework that connects these seemingly disparate domains.

Citations

• Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints
• Probabilistic weather forecasting with machine learning
• Hard Constraint Guided Flow Matching for Gradient-Free Generation of PDE Solutions
• Projected Neural Differential Equations for Learning Constrained Dynamics
• The Elements of Differentiable Programming
• Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs
• Chronos: Learning the Language of Time Series
• Scaling physics-informed hard constraints with mixture-of-experts
• Physics-Informed Neural Networks with Hard Linear Equality Constraints
• Unified Training of Universal Time Series Forecasting Transformers
• A decoder-only foundation model for time-series forecasting
• PreDiff: Precipitation Nowcasting with Latent Diffusion Models
• Automated translation and accelerated solving of differential equations on multiple GPU platforms
• Faithful Heteroscedastic Regression with Neural Networks
• Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems
• Alternating Differentiation for Optimization Layers
• Constants of motion network
• Learning differentiable solvers for systems with hard constraints
• HierarchicalForecast: A Reference Framework for Hierarchical Forecasting in Python
• Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic\n Uncertainty
• Multiwavelet-based Operator Learning for Differential Equations
• Efficient and Modular Implicit Differentiation
• Physics-informed machine learning
• Skilful precipitation nowcasting using deep generative models of radar
• Flexible Model Aggregation for Quantile Regression
• Fourier Neural Operator for Parametric Partial Differential Equations
• MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Universal Differential Equations for Scientific Machine Learning
• Temporal Fusion Transformers for Interpretable Multi-horizon Time Series\n Forecasting
• Differentiable Convex Optimization Layers
• Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators
• High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes
• Reliable training and estimation of variance networks
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Attentive Neural Processes
• Relational inductive biases, deep learning, and graph networks
• A Multi-Horizon Quantile Recurrent Forecaster
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry\n and Semantics
• DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• OptNet: Differentiable Optimization as a Layer in Neural Networks
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
• Выпуклая оптимизация
• Strictly Proper Scoring Rules, Prediction, and Estimation
• Nonlinear total variation based noise removal algorithms
• HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees

Cited by

• Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints

Related

• Neural Measures for learning distributions of Random PDEs
• Probabilistic Neural Networks (PNNs) for Modeling Aleatoric Uncertainty in Scientific Machine Learning
• Calibrated Probabilistic Forecasts for Arbitrary Sequences
• Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models
• Deep Probabilistic Graphical Modeling