Extremal learning: extremizing the output of a neural network in regression problems

2021/02/06 by Zakaria Patel, Markus Rummel, Patel, Zakaria +1
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Model Reduction and Neural Networks #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.2102.03626

openalex publication_date 2021/02/06 · openalex created_date 2021/02/15 · openalex updated_date 2026/07/28

Abstract

Neural networks allow us to model complex relationships between variables. We show how to efficiently find extrema of a trained neural network in regression problems. Finding the extremizing input of an approximated model is formulated as the training of an additional neural network with a loss function that minimizes when the extremizing input is achieved. We further show how to incorporate additional constraints on the input vector such as limiting the extrapolation of the extremizing input vector from the original training data set. An instructional example of this approach using TensorFlow is included.

Citations

• Generative Adversarial Networks
• Long Short-Term Memory
• Gradient-based learning applied to document recognition
• Adversarial Machine Learning at Scale
• Generative Adversarial Networks: An Overview
• Learning Longer-term Dependencies in RNNs with Auxiliary Losses

Related

• Unraveling the Black Box of Neural Networks: A Dynamic Extremum Mapper
• Learning Without Loss
• Robust and Active Learning for Deep Neural Network Regression
• Representation and Regression Problems in Neural Networks: Relaxation, Generalization, and Numerics
• Extrapolation and learning equations